Group sizes vary widely in nature. Our results suggest a reason why: group size is not a fixed advantage or disadvantage. It can impose costs within groups, yet become an asset between groups, with the balance shifting across environmental contexts.