RWKV Math

I evaluated a small RWKV-4 Neo model on longer-digit arithmetic to examine its compositional generalization. The experiment found that the model did not reliably generalize to longer numbers.

The evaluation code is available in the RWKV-4 math evaluation gist. For context and results, see the compositional generalization article.