Put simply, kernel mode setting means the Linux kernel, instead of an X11 video driver, handles the task of configuring the graphics mode of the console. The assignment of this work to the kernel shaves away some overhead, making certain operations faster. It also helps developers by allowing kernel problems to be more easily debugged when X is running, and simplifying the video infrastructure upon which Linux relies.
Normal users shouldn't expect kernel mode setting to revolutionize their Ubuntu experience, but it does bring a few nice improvements, including faster and smoother switching between user accounts and between the X11 console and virtual terminals. This video provides an idea of the new speed of these operations (apologies for the crudeness):
Fast-user switching and access to the virtual console were quite zippy on my i810 video card running in the live environment--and I suspect performance would be even better on an installed system with better hardware (unfortunately, I was only able to test on my old laptop, because the Karmic beta CD wouldn't boot on my desktop; I'm not sure whether kernel mode setting was the reason).
Kernel mode setting also allows virtual terminals (the screens accessible by pressing control-alt-[1-6]) to display at the native resolution of the attached hardware, making them easier to use and less ugly. This change will be especially beneficial to administrators of Ubuntu server edition, where all local interaction with the system has to be done in virtual terminals by default.
There's no official word on when kernel mode setting will be enabled for video chipsets other than Intel, but Ubuntu developers are working on it for some nvidia and ATI devices. Don't expect it in Karmic, though.