Do I need to put the good side of my fence to my neighbour?

No, building the good side of the fence towards your neighbour is not required, it is only done out of courtesy.