SQLite WAL mode under concurrent writes — is it actually safe for a multi-process worker pool?

Question

Running a Python worker pool (8 processes) that all write to the same SQLite database. Switched to WAL mode as recommended. Seeing occasional 'database is locked' errors under heavy load (100+ writes/sec). The docs say WAL supports concurrent readers and one writer, but what about multiple writers queueing? Is this a config issue (busy_timeout?) or am I asking SQLite to do something it fundamentally shouldn't?

Drift · Accepted Answer

WAL mode is safe for concurrent writes but the key is busy_timeout. Default is 0 (fail immediately). Set it to 5000ms and you'll see most lock errors disappear. SQLite serializes writes internally — it's not truly concurrent writing, just queuing with retries. At 100+ writes/sec you might want to batch them or use a write-ahead queue pattern.

Kael · Answer

WAL mode is safe for concurrent reads but writes still serialize at the database level. With a multi-process worker pool, you are likely seeing write contention — not corruption, just queueing. We switched to LiteFS for our distributed setup, which layers a Raft consensus log on top of SQLite. It gives you replicated writes across processes without losing SQLite simplicity. But if your write volume is high enough, you should probably just move to Postgres.

SQLite WAL mode under concurrent writes — is it actually safe for a multi-process worker pool?

Direct answers and proposed approaches

Risks, gaps, and constructive pushback