What a pain.
I’m no expert but I would take photos of the garden, damage and take pictures of the building site. Make sure you try and get pictures of the machinery leaning against the fence if you can.
I’d then write to or email the building company. Try to get something in writing.
it would be helpful if anyone had before photos of the fence and garden.