I've seen amateur radio described as an "expensive hobby." It certainly can be, but when I started in 1959, that never occurred to me. My sources of income in those days were a paper route, mowing lawns ($3 each), a small weekly allowance from my father and cash bonuses for every A grade I got on my report cards (many).  Money was never a problem for me.

I started as an SWL using a Knight Ocean Hopper I built from a kit from Allied Radio in Chicago, which cost only $19 including the case and all five coils. That was less than seven lawns mowed, so not that big a deal. It taught me how to solder & it worked the very first time I turned it on. I immediately discovered the magic of shortwave radio. I still have QSL cards I got from Radio Moscow, VOA, Quito Ecuador and others. But most importantly I discovered ham radio, which started me on my path to having my own ham ticket, KN9AGL, by 1960.

Of course then I bought another kit for my first transmitter, a Heathkit DX-20, which set me back $36, so maybe I had to save up for a while for that. And I also upgraded to a used National NC-109 receiver, which wasn't all that good, but it was fine for 40m CW using the $10 Heathkit Q-Multiplier kit I built. A top of the line Collins ham station in those days could easily cost as much as a new car, but I was already learning to live within my own means in those days.

Starting Out These Days

There are still lots of ways to get started in ham radio without using a ton of money. What follows are some ideas from an Old Timer about how to get a good start in ham radio without spending a lot of money.

1.  Find out whether there is an active ham radio club near you. If there is, join it and attend all their meetings and their Field Day event. This is the single best thing you can do. If you're really lucky, you might even meet someone willing to be your Elmer. Meet a lot of the members and ask a lot of questions. You might even meet some folks with spare rigs they'd like to loan to you, donate to you, or sell to you. This is an especially good strategy for youngsters.
2.  Find out whether there are any local hamfests. If there are any, try to get your Elmer to take you to one and listen to his/her advice about any used equipment that is being offered for sale there.
3.  If you have a local ham radio store nearby (look through all the advertisers in QST), go there and look at any used gear they have for sale. These will cost more than those offered at hamfests, but the store will usually provide some assurance they they checked out the rig and it is working properly. You might also be able to find used gear for sale on their website. 
4.  There are lots of websites where people sell used ham gear. I've bought lots of good stuff on QTH.com, including a very wide range of stuff, some at great prices. But fair warning, looking for good deals on rare old rigs can get pretty addictive. My wife would be happy to show you what is in our basement as proof of that.
5.  My final suggestion is that you look into buying ham gear in kit form. This is especially popular among the QRP community, where smart designers build rigs that squeeze out innovative low power products. I frequently check numerous websites for updates with new products at very affordable prices.  Some notable examples are: Four State QRP Group, Pacific Antenna (QRP Kits), QRP Guys, NM0S Electronics, Kits & Parts, Planet Solar, Kanga Products, QRP Me, Dan's Small Parts & Kits, W8DIZ Kits & Parts, LNR Precision, QRP Labs, SOTA Beams, QRPCI Resources, Phaser Digital Mode Transceivers and Midnight Design Solutions links.  This is not an exhaustive listing, but it could keep you busy for a very long time!