While the official Kodak Playsport replacement battery doesn’t cost that much at ~$20, there are many inexpensive options.  A highly rated alternative is the Kodak KLIC-7004 / Fuji Np-50 / Pentax DL-I68 Compatible Li-Ion Battery, a bargain at ~$2.