I figure I need to post update in case anyone interested.
After removing my brake caliper, I learnt that these suckers (see picture, white arrows) were stuck, hence the caliper cannot move from side to side (green arrow) to center the pads.
So, long story short, I set out to change fork seals and ended up changing brake fluid, new brake pads, a stainless steel brake line.
The forks are looking good, haven't leaked yet !
Took me a week or so to get all the parts. Longest fork seal changing ever !!!
