Spatial modes of light can be used as carriers of information in classical optical communication, or as an alphabet in quantum optical communication. In order to exploit the spatial domain, it is required to (de)multiplex different modes from a shared input channel into different output ports. Mode sorters were already employed in free-space and fiber systems, but so far were not realized for planar guided waves. Here we present a general method for compact on-chip sorting of different planar beams with micrometric footprint and nanometric thickness. The designs are generated using a linkage-tree-based genetic algorithm and are experimentally demonstrated on a surface plasmon-polariton platform by sorting Hermite-Gaussian beams. The method we use here can be readily applied to opti-mize complex, large scale, optical devices involving beam propagation methods.