The recent advance of single-cell technologies has provided an unprecedented opportunity to bring new insights into many complex biological phenomena, such as the regulation of cell differentiation in a multi-cellular organism and cell-to-cell variability in an isogenic population. In this dissertation, we have explored the gene expression regulation using datasets generated by single-cell techniques in three aspects. First, we analyzed a large-scale gene expression dataset measured in individual cells throughout the embryogenesis of C. elegans in a nearly continuous time-scale. We revealed many known and novel genes driving lineage divergence at early cell divisions, facilitating a systematic understanding of the fate specification in C. elegans. Second, we developed a novel clustering algorithm named SNN-Cliq that utilizes the shared nearest neighbor and graph-theoretic partitioning techniques. Our algorithm has the superiority of handling high-dimensional noisy data in that it allows clustering on a variety of single-cell RNA-sequencing (RNA-Seq) data with high accuracy. Last, using an RNA-Seq technique, we profiled transcriptomes in 51 yeast cells from three treatments. Intriguingly, we found that the transcription variation, or noise, shows distinct features under different treatments for certain functional gene modules and regulatory pathways. Our results also suggest that transcriptional noise is subject to regulation in response to environmental stresses. In summary, this dissertation has contributed to algorithmic development for analyzing various single-cell datasets and deepened our knowledge of transcriptional regulation at the single cell level.