How to build windows?

I am trying to build a window into a pre-existing wall, but it says ‘Can’t build there’.
Do I have to have an empty square (and destroy the wall), and is it the same for all construction stuff?

EDIT: And how do I destroy walls if I have to?

You have to deconstruct the wall, and as far as I know you can only do that with a jackhammer.

Or you could ram the wall with a car, preferably a solar one with a seat belt that won’t be missed. I have a hard time finding jackhammers.