Full disclosure - I have no seat time in an E46 M3 but I have owned an E86 Z4MC for the past ~8000 miles and it shares a whole lot.
The S54 is an awesome engine and suited well to MT. Prior to picking up the used coupe I looked at a 997 Carrera S, 987.1 Cayman S, 981 Cayman/Boxster S, Scion FR-S, Miata and S2000. Each of these cars has their own charms and pitfalls, but the E46/E86 platform is really a winning combination of design, performance, limited nannies and collector potential at a reasonable price. It has streetable power (don't really
need more) and is very DIY friendly to help save maintenance costs.
Best of luck!
