Creeper or Peeper?

Last fall the neighbor behind us built a hideous shed that is two stories high with a window on the second “story”.  It is less than 6 feet from our back chain-link fence.  This is bad enough, but I wasn’t going to complain about it.  However, last weekend when the weather warmed up he and another adult friend had chairs up on the second story and were sitting with a perfect vantage point right into our backyard.  This window is only looking down into our backyard and nothing else.  I am very uncomfortable with this.  Is this legal?  First of all, should he have had a permit to build this huge shed, and secondly, is it legal to have this window so close to our fence to look down directly on us? 
Yes, your neighbor probably needed a building permit for his shed.  But provided he complied with setback rules, zoning and building codes I can’t think of anything that would prohibit him from having a window that looked directly into your backyard.  If you decided to, you could put up a barrier that would obstruct his view.  Maybe a few Leyland Cypress.  It is illegal (in North Carolina) for a someone to “peep” secretly into a room occupied by a another person.  So if he used the window for secret peeping he might be committing a crime.  But a backyard is not a “room.”
I can understand why this makes you uncomfortable.  You might try calling a code enforcement department and see if the building had a permit (permits are public record).  If there was no permit and a code inspector found a violation (violating a setback rule, for instance) the inspector could require it to be corrected.