We demonstrate that the size and crystallinity of grains in individual ZnO nanofibers greatly influence their sensing properties for CO. The sensing properties, including sensitivity, response and recovery times of a sensor fabricated with ZnO nanofibers composed of large grains are much superior to those of a sensor fabricated with nanofibers of small grains. The crystallinity, improved by the longer calcination time, is likely to be responsible for the higher sensitivity in the large-grained nanofibers. The facilitated occupancy and desorption of CO molecules at grain boundaries in the large-grained nanofibers are the most probable causes of the shorter response and recovery times in detecting CO, respectively. This work suggests not only that electrospinning-synthesized ZnO nanofibers hold promise for realizing sensitive and reliable gas sensors but also that the size as well as the crystallinity of the grains existing in individual nanofibers need to be optimized to obtain the best sensing properties.