Have you tried a two-pass approach? For example, where prompt #1 is "Which elliptic curves have rational parameterizations?", and then prompt #2 (perhaps to a smaller/faster model like Gemma) is "In the following text, replace all LaTeX-escaped notation with Markdown code blocks and unicode characters. For example, $F_n = F_{n - 1} + F_{n - 2}$ should be replaced with `Fₙ = Fₙ₋₁ + Fₙ₋₂`. <Response from prompt #1>". Although it's not clear how you would want more complex things to be converted.
reStructuredText support :math: roles. AsciiDoc has stem blocks. Why do folks keep trying to shoehorn Markdown into everything, creating yet another fork, when there are other lightweight markup languages that support actual features for technical blogs/documentation?