I am not an expert, so I will not say you are crazy!

But, please do take my words in context, so if I say something you are obviously aware of, please feel free to ignore me.
1. Keep in mind that the MAINS breakers may not be sufficient isolation from the "neighborhood" circuits. The purpose of the transfer switch is to provide true isolation (both live and neutral) so that there is no safety hazard.
2. Consider the legal issue. Imagine what might happen if you fried some local power utility guy who touches the wires outside that he/she thinks are not live.
3. Inside wiring usually has a live and neutral side - some applicances have polarity plugs to ensure that the outside of the cases are connected to the correct side of the electrical feed. If you stick the output of the generator to internal wiring, you need to make sure to maintain the correct polarity to avoid dangerous conditions on your appliances.
4. Do you not have multiple phases of power coming into your house? If yes, what is your plan to handle that? Often, the circuit breakers are connected to different phases (dropping down the 220 to the 110 inside the house by connecting to the correct pairs!)
5. What about 220 to 110 transformers you may have in your house at the meter (for example, dryer and oven feeds) on the 2nd panel? These often have windings that are optimized (primary on the inner core and secondary on the outer) for going "one way", with appropriate cooling, etc. If you feed 110 in to the "output" side, what will these transformers do?
Frankly, with all the possible problems of doing this, I think I would not mind the "extension cord through the windows" problem you want to avoid! I'd rather be safe than sorry!
Please be safe, whatever you do.
Z