Not surprisingly, the layers in the ice are thinner as you drill deeper. The lower layers are under a lot of pressure, and they've been there longer. Ice gets added on the top of an icecap, by snowstorms. But ice leaves by being squeezed out to the sides. The sides break off, and become icebergs.
Eventually, the drill gets to a depth where the layers can't be counted any more. In Greenland, the GISP2 project couldn't do direct, accurate counts below layer 50,000. However, there are reasons to think the bottom ice, 3000 meters down, is 250,000 years old.
In the Antarctic icecap, the "Vostok" core got to a depth of 3623 meters in 1998. I believe the Russian/French team plan to eventually drill another 120 meters.
See, for example,
Climate and atmospheric history of the past 420,000 years from the Vostok ice core, Antarctica, Petit et al, Nature Vol. 399 pp. 429-436, 3 June 1999
Climate correlations between Greenland and Antarctica during the past 100,000 years, Bender et al, Nature Vol. 372 pp. 663-666, 15 December 1994
Evidence for general instability of past climate from a 250-kyr ice core record, Dansgaard et al, Nature Vol. 364 pp. 218-220 15 July 1993
Extending the Vostok ice-core record of palaeoclimate to the penultimate glacial period, Jouzel et al, Nature Vol. 364 pp. 407-412, 29 July 1993