> so it doesn't generally let code directly manipulate the call stack.
You are doing something wrong if you need to manipulate the call stack to do a tail call. You just have to update the local arguments and jump back to the start of the function. It is literally just turning the call into a loop.
That works for tail recursive functions, but not tail calls.
TCO works for all tail calls, not just the recursive ones, so that jump would be to another function's body. The current stack frame may not have enough space for the new function's.
It's true, but you probably don't want TCO for all tail calls since you lose information in stack traces.
As long as you have control over all the code generated you can in principle compile multiple functions into a single one and just add some stubs for interop purposes. This of course complicates how you track function pointers (or equivalently) and may have some overhead.
Still the cases where you can use recur today could be done automatically by the compiler without much trouble.
One worry I have about automatic TCO where recur/loop currently is used is with Lisp's dynamic nature. What happens to everyone who has a reference to the fn when you redefine its def?
You're basically switching (defn f [] (recur [] (loop))) to (defn f [] (f)). What if I do (def g f) and then (def f +)? Will the original f now recur into + when I call (g)? If it still calls into the original f then that creates a mismatch between reloading normal and recur'd functions adding complexity to a language striving for simplicity.
If you want late binding at the call site it's probably unavoidable to have some extra complexity anyways, what if I changed f into something that wasn't callable?
However Clojure binds at the definition time, e.g.
> (defn f [] 42)
#'sandbox9516/f
> (f)
42
> (def g f)
#'sandbox9516/g
> (g)
42
> (def f 1337)
#'sandbox9516/f
> (g)
42
> (f)
java.lang.ClassCastException: java.lang.Long cannot be cast to clojure.lang.IFn
That's because defn binds f to the _value_ of the function, while (f) binds to the _variable_.
> (defn f [] (f))
#'sandbox9672/f
> (def g f)
#'sandbox9672/g
> (def f 42)
#'sandbox9672/f
> (g)
java.lang.ClassCastException: java.lang.Long cannot be cast to clojure.lang.IFn
https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-6.ht...
> so it doesn't generally let code directly manipulate the call stack.
You are doing something wrong if you need to manipulate the call stack to do a tail call. You just have to update the local arguments and jump back to the start of the function. It is literally just turning the call into a loop.