"Atari: 80 Classic Games In One" is a really wonderful CD-ROM game that shouldn't cost more than $20. A little more than 60 are actual 2600 games, the rest are arcade versions of hits like Asteroids, Centipede, etc.
"Activision Anthology" is a really great game for Game Boy Advance because it's really hard to find ROMs made by Activision. This has over 50 Atari 2600 games, including Pitfall, Pitfall 2, Pressure Cooker, and more. It also has a couple of homebrew games like Oystron and Climber 5, and a few unnamed prototypes and the venetian blinds demo. This shouldn't cost more than $20-25.
Also, Atari is making those Plug and Play consoles. You don't need a system or anything, and they have about 20 games built in. I think the thing looks like a model of a 2600, and it has a retro controller. These run about $20-30.