No, you cannot shoot your movies where ever you want. In fact, part of what makes filmmaking such a costly industry are the permits and insurance, as well as the endless paperwork involved. Studios can obviously afford these expenses; amateur independent filmmakers usually cannot. The reason why the studios refuse to "wing" a project and save their money is because permits and insurance lower the risk of legal of troubles, which can put an entire studio, as well as jobs, at stake. That's not to imply that independent filmmakers have less to lose (sort of). But they have the benefit of being less noticeable than a studio.

A movie requires insurance if it is to be shot on location outside of studio-owned space. If an accident occurs, or if any party is injured through the shoot, then the movie and its financiers can be held accountable. The severity of these accidents can range from negligible to tragic, and the purpose of insurance is to offset the chance of legal action against the project as a whole. Thus, if anything happens, the "show must go on," as they say. Without insurance, shooting may be suspended indefinitely. Permits compliment insurance. Since it is shot on location, it can be assumed that the location resides on land owned by the city or state. Permits are valid declarations of permission issued by the city/state; it gives the cast and crew (and equipment) legal assurance to use the desired location, without direct interference or harassment from officials and law enforcement. If shot in public with a heavy pedestrian presence, then it is the crew's job to keep the general public from interfering, because outside help from city officials is never guaranteed.

Now if you're an independent filmmaker with very little cash to spare (or have very little know-how in legal matters), ignore the last paragraph. Permits and insurance are very important; vital, even. But depending on the size of your cast crew, and the scope of your project, guerilla filmmaking may be the best option. Again, city officials and law enforcement are generally willing to turn a blind eye on a tame, pint-sized shoot. But it's also good common sense to at least acquire permission on a personal level from the owner of the location (a house or restaurant, for example). Apart from that philosophy, keep in mind that guerilla filmmaking is about maintaining the public around you (or avoiding them). Be manipulative and sensitive at the same time. Know where to point the camera and where to avert it, as people like their privacy. And if push comes to shove, shoot fast and get out of there ASAP.

The end result may not be what you had in mind; or maybe it is. On both a technical and stylistic level, hastily shot and mostly spontaneous shoots have a jittery, erratic tone. Let this be another word of advice: fast, cinema-verite filmmaking isn't so much technically inferior to polished studio shoots. It's just a different brand, and one that requires knowledge beforehand to be properly taken advantage of.