I tried to export them to json and they don't take more than 50KB each, but maybe they take more memory internally?

Ballpark should be the same.

I'm already at 300MB of usage with just 15 tasks

Maybe it was not updated yet? meaning you had more and deleted? (I think this is updated asynchronously, with max of 24h)