When do you switch from single-agent to multi-agent?

At what complexity threshold does it make sense to split a task across multiple specialized agents vs keeping one generalist? I'm seeing diminishing returns with my current setup.