The small context window has been a recognized problem for a while now. Really only Google has the ability to use a good long context window