In this talk, I will present a new approach for asymptotic expansion of a Markov semigroup with respect to a small parameter. Our idea combines the parametrix method and stochastic calculus of variations. As an application, we deduce the small time and the small noise heat kernel expansions of some hypo-elliptic processes. We will also derive more tractable expansion for the density of such processes and discuss extensions to some degenerate Kolmogorov equations and skew diffusions. (Joint work with Arturo Kohatsu-Higa.)