Part of the renovation of my townhome involves replacing the old ceramic floors with wood ones. The old floors were cemented to the concrete slab and when they were removed they left quite a bit of residue mortar. The most reasonable way to get that mortar off was to grind it with a concrete surfacer with 40 grit diamonds. These gadgets are very large and heavy and have to be disassembled for transport. However, they make short work out of a nasty job. See photos below.

You can see the rough surface from the old mortar.
Wearing a respirator is a must!
Nice and smooth!