This article describes the facile and scalable synthesis of carbon xerogel–bismuth nanoparticle composites using two different approaches. It also demonstrates the high potential of these materials for developing electrochemical sensors, which could simultaneously analyze in a rapid test very low concentrations (<1 ppb) of heavy metals in water. The microstructural characterization of the composites by different techniques revealed a microporous carbon structure with evenly dispersed spherical Bi nanoparticles whose dimensions depend on the synthesis conditions. Sensors prepared with the nanocomposites were used to test the electrochemical performance of the materials for the detection of several heavy metal ions in water such as cadmium and lead, which are included in the list of priority substances of most water policies. The minimum concentration detected for these two species was 0.6 ppb for an overall analysis time of below 5 min. This concentration is well below the maximum limits allowed in drinking water according to the most stringent regulations, and of the order of the maximum allowance concentration of environmental quality standards.