Incredible job. Feels dumb or obvious to say this, but this really changes the way I think of using it. The slow autoregression really sucks because it inhibits your ability to skim sections. For me, that creates an unnatural reading environment. This makes chatgpt feel antiqued.
Yes, agreed. We believe the benefits of reducing latency are non-linear. You can hit different phase changes as the latency reduces and new applications become viable. Roundtripping text-to-speech and speech-to-text is one example. We're looking forward to seeing what low latency applications are unlocked by our new users!
