If I buy a set of rims that have the same bolt pattern as the stock alloys, can I use the lugs that I already have, or should I go with factory specific lugs? I got quoted from Toyota that it would be $140 and change to get a full set of new lugs for the rims I want...

go to a local store that does like brakes n tires, and if they sell rims, you should be able to get them for like 35 bucks...i did at least.

a rim shop would try to rip you as bad as toyota.