Spin glasses can be thought of as complex systems as they have many degrees of freedom and many competing energy scales. Despite of the complex structure of spin glasses methods have been developed which allow some analytical analysis, which frequently is impossible in other complex systems. The free energy and the specific heat of the two-dimensional Gaussian random bond Ising model on a square lattice are found with high accuracy using graph expansion method. At low temperatures the specific heat reveals a zero temperature criticality described by the power law C= constant T^{1+alpha} , with alpha = 0.55(8). Interpretation of the free energy in terms of independent two-level excitations gives the density of states, that follows a novel power law rho(epsilon)= constant epsilon^alpha$ at low energies. An exact high-temperature series for this model up to the term beta^ 29 is found. A proof that the density of one-site spin flip states vanishes at low energy is given.