The unit commitment problem (UCP) for thermal units consists of finding an optimal electricity production plan for a given time horizon. In this paper we propose hybrid approaches which combine Variable Neighborhood Search (VNS) metaheuristic and mathematical programming to solve this NP-hard problem. Four new VNS based methods, including one with adaptive choice of neighborhood order used within deterministic exploration of neighborhoods, are proposed. A convex economic dispatch subproblem is solved by Lambda iteration method in each time period. Extensive computational experiments are performed on well-known test instances from the literature as well as on new large instances generated by us. It appears that the proposed heuristics successfully solve both small and large scale problems. Moreover, they outperform other well-known heuristics that can be considered as the state-of-the-art approaches.