[{"data":1,"prerenderedAt":1300},["ShallowReactive",2],{"page-\u002Fasync-engines-dialects-and-connection-pooling\u002Fchoosing-between-asyncpg-and-psycopg-async-drivers\u002F":3},{"id":4,"title":5,"body":6,"description":1293,"extension":1294,"meta":1295,"navigation":198,"path":1296,"seo":1297,"stem":1298,"__hash__":1299},"content\u002Fasync-engines-dialects-and-connection-pooling\u002Fchoosing-between-asyncpg-and-psycopg-async-drivers\u002Findex.md","Choosing Between asyncpg and psycopg Async Drivers",{"type":7,"value":8,"toc":1282},"minimark",[9,13,22,31,36,39,54,84,104,108,111,125,139,319,331,335,345,352,357,561,566,736,762,766,769,776,781,955,958,962,965,982,989,993,1002,1007,1047,1050,1064,1068,1071,1101,1104,1107,1110,1114,1196,1200,1215,1238,1270,1278],[10,11,5],"h1",{"id":12},"choosing-between-asyncpg-and-psycopg-async-drivers",[14,15,16,17,21],"p",{},"When architecting high-throughput, non-blocking Python services, selecting the correct PostgreSQL driver is a foundational infrastructure decision. The ",[18,19,20],"code",{},"asyncpg vs psycopg async driver"," debate centers on execution models, protocol efficiency, and ecosystem compatibility. SQLAlchemy 2.0’s modern async API provides a unified abstraction layer, but understanding the underlying driver mechanics is critical for avoiding event loop starvation, optimizing connection lifecycle, and achieving predictable latency.",[14,23,24,25,30],{},"For a comprehensive overview of how SQLAlchemy 2.0 manages asynchronous execution contexts, refer to the foundational concepts in ",[26,27,29],"a",{"href":28},"\u002Fasync-engines-dialects-and-connection-pooling\u002F","Async Engines, Dialects, and Connection Pooling",".",[32,33,35],"h2",{"id":34},"architectural-differences-and-async-execution-models","Architectural Differences and Async Execution Models",[14,37,38],{},"The core divergence lies in how each driver interfaces with PostgreSQL and the Python event loop.",[14,40,41,45,46,49,50,53],{},[42,43,44],"strong",{},"asyncpg"," implements a pure-Python, native async protocol. It bypasses ",[18,47,48],{},"libpq"," entirely, communicating directly with PostgreSQL via the frontend\u002Fbackend wire protocol. This design eliminates the overhead of synchronous C-extension wrappers and provides zero-copy binary data serialization. Coroutine scheduling is handled natively by ",[18,51,52],{},"asyncio",", meaning I\u002FO operations yield control precisely at network boundaries without thread pool fallbacks.",[14,55,56,59,60,63,64,66,67,69,70,72,73,75,76,79,80,83],{},[42,57,58],{},"psycopg3"," (specifically ",[18,61,62],{},"psycopg[async]",") provides async bindings over the battle-tested ",[18,65,48],{}," C library. While it exposes ",[18,68,52],{},"-compatible APIs, it relies on ",[18,71,48],{},"'s internal state machine and uses ",[18,74,52],{},"'s ",[18,77,78],{},"loop.add_reader()","\u002F",[18,81,82],{},"add_writer()"," to monitor socket readiness. This introduces a slight abstraction penalty but guarantees strict compliance with PostgreSQL's official client protocol and inherits decades of connection stability guarantees.",[14,85,86,89,90,92,93,95,96,99,100,103],{},[42,87,88],{},"Thread-Safety & Scheduling Overhead:"," ",[18,91,44],{}," is strictly single-threaded per connection, enforcing a clear async\u002Fawait boundary that prevents accidental blocking. ",[18,94,58],{}," async connections are also event-loop bound, but legacy patterns that mix synchronous ",[18,97,98],{},"psycopg2"," calls or use ",[18,101,102],{},"run_in_executor()"," for DB operations will immediately block the loop. Production systems must enforce strict coroutine boundaries: never call synchronous ORM methods or raw driver functions from an async context without explicit thread delegation.",[32,105,107],{"id":106},"query-execution-patterns-and-performance-tuning","Query Execution Patterns and Performance Tuning",[14,109,110],{},"Throughput characteristics differ significantly based on protocol implementation and caching strategies.",[14,112,113,89,116,118,119,121,122,124],{},[42,114,115],{},"Binary vs Text Protocol:",[18,117,44],{}," defaults to the PostgreSQL binary protocol, which eliminates string parsing overhead for numeric, timestamp, and JSONB types. Under high-concurrency read\u002Fwrite workloads, this typically yields 15–30% lower latency for complex joins and bulk operations. ",[18,120,58],{}," defaults to text protocol but can be configured for binary transmission; however, type adaptation remains slightly heavier due to ",[18,123,48],{},"'s intermediate parsing layer.",[14,126,127,89,130,132,133,135,136,138],{},[42,128,129],{},"Prepared Statement Caching:",[18,131,44],{}," maintains an LRU statement cache at the driver level, automatically preparing and reusing execution plans for repeated queries. ",[18,134,58],{}," relies on PostgreSQL's server-side prepared statements or ",[18,137,48],{},"'s client-side cache, which requires more explicit tuning to avoid cache bloat.",[140,141,146],"pre",{"className":142,"code":143,"language":144,"meta":145,"style":145},"language-python shiki shiki-themes github-light github-dark","from sqlalchemy import text\nfrom sqlalchemy.ext.asyncio import AsyncSession\nfrom typing import List, Dict\n\nasync def execute_bulk_insert(session: AsyncSession, records: List[Dict]) -> None:\n \"\"\"\n Demonstrates SQLAlchemy 2.0 text() execution with explicit bind parameters.\n Both drivers will cache the execution plan, but asyncpg's LRU cache \n activates automatically after the first execution.\n \"\"\"\n query = text(\n \"INSERT INTO audit_logs (event_id, payload, created_at) \"\n \"VALUES (:event_id, :payload, :created_at) \"\n \"ON CONFLICT (event_id) DO UPDATE SET payload = EXCLUDED.payload\"\n )\n \n # execute() accepts a list of dicts for bulk insertion\n await session.execute(query, records)\n await session.commit()\n","python","",[18,147,148,167,180,193,200,223,230,236,242,248,253,265,271,277,283,289,295,302,311],{"__ignoreMap":145},[149,150,153,157,161,164],"span",{"class":151,"line":152},"line",1,[149,154,156],{"class":155},"szBVR","from",[149,158,160],{"class":159},"sVt8B"," sqlalchemy ",[149,162,163],{"class":155},"import",[149,165,166],{"class":159}," text\n",[149,168,170,172,175,177],{"class":151,"line":169},2,[149,171,156],{"class":155},[149,173,174],{"class":159}," sqlalchemy.ext.asyncio ",[149,176,163],{"class":155},[149,178,179],{"class":159}," AsyncSession\n",[149,181,183,185,188,190],{"class":151,"line":182},3,[149,184,156],{"class":155},[149,186,187],{"class":159}," typing ",[149,189,163],{"class":155},[149,191,192],{"class":159}," List, Dict\n",[149,194,196],{"class":151,"line":195},4,[149,197,199],{"emptyLinePlaceholder":198},true,"\n",[149,201,203,206,209,213,216,220],{"class":151,"line":202},5,[149,204,205],{"class":155},"async",[149,207,208],{"class":155}," def",[149,210,212],{"class":211},"sScJk"," execute_bulk_insert",[149,214,215],{"class":159},"(session: AsyncSession, records: List[Dict]) -> ",[149,217,219],{"class":218},"sj4cs","None",[149,221,222],{"class":159},":\n",[149,224,226],{"class":151,"line":225},6,[149,227,229],{"class":228},"sZZnC"," \"\"\"\n",[149,231,233],{"class":151,"line":232},7,[149,234,235],{"class":228}," Demonstrates SQLAlchemy 2.0 text() execution with explicit bind parameters.\n",[149,237,239],{"class":151,"line":238},8,[149,240,241],{"class":228}," Both drivers will cache the execution plan, but asyncpg's LRU cache \n",[149,243,245],{"class":151,"line":244},9,[149,246,247],{"class":228}," activates automatically after the first execution.\n",[149,249,251],{"class":151,"line":250},10,[149,252,229],{"class":228},[149,254,256,259,262],{"class":151,"line":255},11,[149,257,258],{"class":159}," query ",[149,260,261],{"class":155},"=",[149,263,264],{"class":159}," text(\n",[149,266,268],{"class":151,"line":267},12,[149,269,270],{"class":228}," \"INSERT INTO audit_logs (event_id, payload, created_at) \"\n",[149,272,274],{"class":151,"line":273},13,[149,275,276],{"class":228}," \"VALUES (:event_id, :payload, :created_at) \"\n",[149,278,280],{"class":151,"line":279},14,[149,281,282],{"class":228}," \"ON CONFLICT (event_id) DO UPDATE SET payload = EXCLUDED.payload\"\n",[149,284,286],{"class":151,"line":285},15,[149,287,288],{"class":159}," )\n",[149,290,292],{"class":151,"line":291},16,[149,293,294],{"class":159}," \n",[149,296,298],{"class":151,"line":297},17,[149,299,301],{"class":300},"sJ8bj"," # execute() accepts a list of dicts for bulk insertion\n",[149,303,305,308],{"class":151,"line":304},18,[149,306,307],{"class":155}," await",[149,309,310],{"class":159}," session.execute(query, records)\n",[149,312,314,316],{"class":151,"line":313},19,[149,315,307],{"class":155},[149,317,318],{"class":159}," session.commit()\n",[14,320,321,324,325,327,328,330],{},[42,322,323],{},"Performance Trade-off:"," If your workload consists of highly dynamic, ad-hoc queries with low repetition, ",[18,326,58],{},"'s lighter cache footprint may reduce memory pressure. For predictable, parameterized API backends, ",[18,329,44],{},"'s aggressive caching delivers superior sustained throughput.",[32,332,334],{"id":333},"connection-pool-configuration-and-resource-management","Connection Pool Configuration and Resource Management",[14,336,337,338,340,341,344],{},"Connection pooling in async SQLAlchemy requires careful tuning to prevent exhaustion during traffic spikes. While ",[18,339,44],{}," ships with its own native pool, SQLAlchemy's ",[18,342,343],{},"AsyncAdaptedQueuePool"," provides a standardized, dialect-agnostic interface that integrates seamlessly with the ORM.",[14,346,347,348,30],{},"For deep dives into pool lifecycle tuning, see ",[26,349,351],{"href":350},"\u002Fasync-engines-dialects-and-connection-pooling\u002Fconfiguring-async-engines-and-connection-pools\u002F","Configuring Async Engines and Connection Pools",[14,353,354],{},[42,355,356],{},"asyncpg Engine Initialization:",[140,358,360],{"className":142,"code":359,"language":144,"meta":145,"style":145},"import os\nfrom sqlalchemy.ext.asyncio import create_async_engine, async_sessionmaker, AsyncSession\n\nDATABASE_URL = os.getenv(\"DATABASE_URL\", \"postgresql+asyncpg:\u002F\u002Fapp:secret@db:5432\u002Fproduction\")\n\nasync_engine = create_async_engine(\n DATABASE_URL,\n pool_size=20, # Core connections kept alive\n max_overflow=10, # Temporary burst capacity\n pool_recycle=1800, # Reclaim connections after 30m to avoid server-side timeouts\n pool_pre_ping=True, # Validate connection before checkout\n echo=False, # Disable in production to reduce I\u002FO overhead\n)\n\nAsyncSessionFactory = async_sessionmaker(\n bind=async_engine,\n class_=AsyncSession,\n expire_on_commit=False, # Prevent lazy-load blocking in async contexts\n)\n",[18,361,362,369,380,384,407,411,421,429,445,460,475,490,505,509,513,523,533,543,557],{"__ignoreMap":145},[149,363,364,366],{"class":151,"line":152},[149,365,163],{"class":155},[149,367,368],{"class":159}," os\n",[149,370,371,373,375,377],{"class":151,"line":169},[149,372,156],{"class":155},[149,374,174],{"class":159},[149,376,163],{"class":155},[149,378,379],{"class":159}," create_async_engine, async_sessionmaker, AsyncSession\n",[149,381,382],{"class":151,"line":182},[149,383,199],{"emptyLinePlaceholder":198},[149,385,386,389,392,395,398,401,404],{"class":151,"line":195},[149,387,388],{"class":218},"DATABASE_URL",[149,390,391],{"class":155}," =",[149,393,394],{"class":159}," os.getenv(",[149,396,397],{"class":228},"\"DATABASE_URL\"",[149,399,400],{"class":159},", ",[149,402,403],{"class":228},"\"postgresql+asyncpg:\u002F\u002Fapp:secret@db:5432\u002Fproduction\"",[149,405,406],{"class":159},")\n",[149,408,409],{"class":151,"line":202},[149,410,199],{"emptyLinePlaceholder":198},[149,412,413,416,418],{"class":151,"line":225},[149,414,415],{"class":159},"async_engine ",[149,417,261],{"class":155},[149,419,420],{"class":159}," create_async_engine(\n",[149,422,423,426],{"class":151,"line":232},[149,424,425],{"class":218}," DATABASE_URL",[149,427,428],{"class":159},",\n",[149,430,431,435,437,440,442],{"class":151,"line":238},[149,432,434],{"class":433},"s4XuR"," pool_size",[149,436,261],{"class":155},[149,438,439],{"class":218},"20",[149,441,400],{"class":159},[149,443,444],{"class":300},"# Core connections kept alive\n",[149,446,447,450,452,455,457],{"class":151,"line":244},[149,448,449],{"class":433}," max_overflow",[149,451,261],{"class":155},[149,453,454],{"class":218},"10",[149,456,400],{"class":159},[149,458,459],{"class":300},"# Temporary burst capacity\n",[149,461,462,465,467,470,472],{"class":151,"line":250},[149,463,464],{"class":433}," pool_recycle",[149,466,261],{"class":155},[149,468,469],{"class":218},"1800",[149,471,400],{"class":159},[149,473,474],{"class":300},"# Reclaim connections after 30m to avoid server-side timeouts\n",[149,476,477,480,482,485,487],{"class":151,"line":255},[149,478,479],{"class":433}," pool_pre_ping",[149,481,261],{"class":155},[149,483,484],{"class":218},"True",[149,486,400],{"class":159},[149,488,489],{"class":300},"# Validate connection before checkout\n",[149,491,492,495,497,500,502],{"class":151,"line":267},[149,493,494],{"class":433}," echo",[149,496,261],{"class":155},[149,498,499],{"class":218},"False",[149,501,400],{"class":159},[149,503,504],{"class":300},"# Disable in production to reduce I\u002FO overhead\n",[149,506,507],{"class":151,"line":273},[149,508,406],{"class":159},[149,510,511],{"class":151,"line":279},[149,512,199],{"emptyLinePlaceholder":198},[149,514,515,518,520],{"class":151,"line":285},[149,516,517],{"class":159},"AsyncSessionFactory ",[149,519,261],{"class":155},[149,521,522],{"class":159}," async_sessionmaker(\n",[149,524,525,528,530],{"class":151,"line":291},[149,526,527],{"class":433}," bind",[149,529,261],{"class":155},[149,531,532],{"class":159},"async_engine,\n",[149,534,535,538,540],{"class":151,"line":297},[149,536,537],{"class":433}," class_",[149,539,261],{"class":155},[149,541,542],{"class":159},"AsyncSession,\n",[149,544,545,548,550,552,554],{"class":151,"line":304},[149,546,547],{"class":433}," expire_on_commit",[149,549,261],{"class":155},[149,551,499],{"class":218},[149,553,400],{"class":159},[149,555,556],{"class":300},"# Prevent lazy-load blocking in async contexts\n",[149,558,559],{"class":151,"line":313},[149,560,406],{"class":159},[14,562,563],{},[42,564,565],{},"psycopg3 Async Engine Configuration:",[140,567,569],{"className":142,"code":568,"language":144,"meta":145,"style":145},"import ssl\nfrom sqlalchemy.ext.asyncio import create_async_engine\n\nPSYCOPG_URL = \"postgresql+psycopg:\u002F\u002Fapp:secret@db:5432\u002Fproduction\"\n\n# Inject production-grade SSL context\nssl_ctx = ssl.create_default_context(cafile=\"\u002Fetc\u002Fssl\u002Fcerts\u002Fca-bundle.crt\")\nconnect_args = {\"ssl\": ssl_ctx, \"connect_timeout\": 5}\n\nasync_engine_psycopg = create_async_engine(\n PSYCOPG_URL,\n connect_args=connect_args,\n pool_size=15,\n max_overflow=5,\n pool_recycle=3600,\n pool_pre_ping=True,\n)\n",[18,570,571,578,589,593,603,607,612,632,660,664,673,680,690,701,711,722,732],{"__ignoreMap":145},[149,572,573,575],{"class":151,"line":152},[149,574,163],{"class":155},[149,576,577],{"class":159}," ssl\n",[149,579,580,582,584,586],{"class":151,"line":169},[149,581,156],{"class":155},[149,583,174],{"class":159},[149,585,163],{"class":155},[149,587,588],{"class":159}," create_async_engine\n",[149,590,591],{"class":151,"line":182},[149,592,199],{"emptyLinePlaceholder":198},[149,594,595,598,600],{"class":151,"line":195},[149,596,597],{"class":218},"PSYCOPG_URL",[149,599,391],{"class":155},[149,601,602],{"class":228}," \"postgresql+psycopg:\u002F\u002Fapp:secret@db:5432\u002Fproduction\"\n",[149,604,605],{"class":151,"line":202},[149,606,199],{"emptyLinePlaceholder":198},[149,608,609],{"class":151,"line":225},[149,610,611],{"class":300},"# Inject production-grade SSL context\n",[149,613,614,617,619,622,625,627,630],{"class":151,"line":232},[149,615,616],{"class":159},"ssl_ctx ",[149,618,261],{"class":155},[149,620,621],{"class":159}," ssl.create_default_context(",[149,623,624],{"class":433},"cafile",[149,626,261],{"class":155},[149,628,629],{"class":228},"\"\u002Fetc\u002Fssl\u002Fcerts\u002Fca-bundle.crt\"",[149,631,406],{"class":159},[149,633,634,637,639,642,645,648,651,654,657],{"class":151,"line":238},[149,635,636],{"class":159},"connect_args ",[149,638,261],{"class":155},[149,640,641],{"class":159}," {",[149,643,644],{"class":228},"\"ssl\"",[149,646,647],{"class":159},": ssl_ctx, ",[149,649,650],{"class":228},"\"connect_timeout\"",[149,652,653],{"class":159},": ",[149,655,656],{"class":218},"5",[149,658,659],{"class":159},"}\n",[149,661,662],{"class":151,"line":244},[149,663,199],{"emptyLinePlaceholder":198},[149,665,666,669,671],{"class":151,"line":250},[149,667,668],{"class":159},"async_engine_psycopg ",[149,670,261],{"class":155},[149,672,420],{"class":159},[149,674,675,678],{"class":151,"line":255},[149,676,677],{"class":218}," PSYCOPG_URL",[149,679,428],{"class":159},[149,681,682,685,687],{"class":151,"line":267},[149,683,684],{"class":433}," connect_args",[149,686,261],{"class":155},[149,688,689],{"class":159},"connect_args,\n",[149,691,692,694,696,699],{"class":151,"line":273},[149,693,434],{"class":433},[149,695,261],{"class":155},[149,697,698],{"class":218},"15",[149,700,428],{"class":159},[149,702,703,705,707,709],{"class":151,"line":279},[149,704,449],{"class":433},[149,706,261],{"class":155},[149,708,656],{"class":218},[149,710,428],{"class":159},[149,712,713,715,717,720],{"class":151,"line":285},[149,714,464],{"class":433},[149,716,261],{"class":155},[149,718,719],{"class":218},"3600",[149,721,428],{"class":159},[149,723,724,726,728,730],{"class":151,"line":291},[149,725,479],{"class":433},[149,727,261],{"class":155},[149,729,484],{"class":218},[149,731,428],{"class":159},[149,733,734],{"class":151,"line":297},[149,735,406],{"class":159},[14,737,738,741,742,745,746,749,750,753,754,757,758,761],{},[42,739,740],{},"Resource Management Strategy:"," Always pair ",[18,743,744],{},"pool_size"," with your application's concurrency limit. If your ASGI server handles 100 concurrent requests, setting ",[18,747,748],{},"pool_size=20"," and ",[18,751,752],{},"max_overflow=10"," ensures that excess requests queue gracefully instead of triggering ",[18,755,756],{},"ConnectionTimeoutError",". Enable ",[18,759,760],{},"pool_pre_ping"," in cloud environments where idle TCP connections are silently dropped by load balancers.",[32,763,765],{"id":764},"framework-integration-and-lifecycle-management","Framework Integration and Lifecycle Management",[14,767,768],{},"Modern web frameworks require deterministic startup\u002Fshutdown hooks to bind engine lifecycles to the application process. Dependency injection scopes should align with request boundaries to prevent cross-request transaction leakage.",[14,770,771,772,30],{},"Implementation patterns for request-scoped sessions are detailed in ",[26,773,775],{"href":774},"\u002Fasync-engines-dialects-and-connection-pooling\u002Fintegrating-sqlalchemy-async-with-fastapi-and-starlette\u002F","Integrating SQLAlchemy Async with FastAPI and Starlette",[14,777,778],{},[42,779,780],{},"Async Session Context Manager Pattern:",[140,782,784],{"className":142,"code":783,"language":144,"meta":145,"style":145},"from contextlib import asynccontextmanager\nfrom typing import AsyncGenerator\nfrom sqlalchemy.ext.asyncio import AsyncSession\n\n@asynccontextmanager\nasync def get_db_session() -> AsyncGenerator[AsyncSession, None]:\n \"\"\"\n Production-ready transactional scope.\n Automatically commits on success, rolls back on exception,\n and guarantees connection release back to the pool.\n \"\"\"\n async with AsyncSessionFactory() as session:\n try:\n yield session\n await session.commit()\n except Exception:\n await session.rollback()\n raise\n finally:\n # Explicit close ensures the connection is returned to the pool\n # even if the generator is garbage collected prematurely.\n await session.close()\n",[18,785,786,798,809,819,823,828,845,849,854,859,864,868,885,892,900,906,916,923,928,935,941,947],{"__ignoreMap":145},[149,787,788,790,793,795],{"class":151,"line":152},[149,789,156],{"class":155},[149,791,792],{"class":159}," contextlib ",[149,794,163],{"class":155},[149,796,797],{"class":159}," asynccontextmanager\n",[149,799,800,802,804,806],{"class":151,"line":169},[149,801,156],{"class":155},[149,803,187],{"class":159},[149,805,163],{"class":155},[149,807,808],{"class":159}," AsyncGenerator\n",[149,810,811,813,815,817],{"class":151,"line":182},[149,812,156],{"class":155},[149,814,174],{"class":159},[149,816,163],{"class":155},[149,818,179],{"class":159},[149,820,821],{"class":151,"line":195},[149,822,199],{"emptyLinePlaceholder":198},[149,824,825],{"class":151,"line":202},[149,826,827],{"class":211},"@asynccontextmanager\n",[149,829,830,832,834,837,840,842],{"class":151,"line":225},[149,831,205],{"class":155},[149,833,208],{"class":155},[149,835,836],{"class":211}," get_db_session",[149,838,839],{"class":159},"() -> AsyncGenerator[AsyncSession, ",[149,841,219],{"class":218},[149,843,844],{"class":159},"]:\n",[149,846,847],{"class":151,"line":232},[149,848,229],{"class":228},[149,850,851],{"class":151,"line":238},[149,852,853],{"class":228}," Production-ready transactional scope.\n",[149,855,856],{"class":151,"line":244},[149,857,858],{"class":228}," Automatically commits on success, rolls back on exception,\n",[149,860,861],{"class":151,"line":250},[149,862,863],{"class":228}," and guarantees connection release back to the pool.\n",[149,865,866],{"class":151,"line":255},[149,867,229],{"class":228},[149,869,870,873,876,879,882],{"class":151,"line":267},[149,871,872],{"class":155}," async",[149,874,875],{"class":155}," with",[149,877,878],{"class":159}," AsyncSessionFactory() ",[149,880,881],{"class":155},"as",[149,883,884],{"class":159}," session:\n",[149,886,887,890],{"class":151,"line":273},[149,888,889],{"class":155}," try",[149,891,222],{"class":159},[149,893,894,897],{"class":151,"line":279},[149,895,896],{"class":155}," yield",[149,898,899],{"class":159}," session\n",[149,901,902,904],{"class":151,"line":285},[149,903,307],{"class":155},[149,905,318],{"class":159},[149,907,908,911,914],{"class":151,"line":291},[149,909,910],{"class":155}," except",[149,912,913],{"class":218}," Exception",[149,915,222],{"class":159},[149,917,918,920],{"class":151,"line":297},[149,919,307],{"class":155},[149,921,922],{"class":159}," session.rollback()\n",[149,924,925],{"class":151,"line":304},[149,926,927],{"class":155}," raise\n",[149,929,930,933],{"class":151,"line":313},[149,931,932],{"class":155}," finally",[149,934,222],{"class":159},[149,936,938],{"class":151,"line":937},20,[149,939,940],{"class":300}," # Explicit close ensures the connection is returned to the pool\n",[149,942,944],{"class":151,"line":943},21,[149,945,946],{"class":300}," # even if the generator is garbage collected prematurely.\n",[149,948,950,952],{"class":151,"line":949},22,[149,951,307],{"class":155},[149,953,954],{"class":159}," session.close()\n",[14,956,957],{},"Middleware interception for OpenTelemetry tracing or Prometheus metrics should wrap the session yield, capturing query duration and connection checkout latency without modifying business logic.",[32,959,961],{"id":960},"background-task-processing-and-worker-compatibility","Background Task Processing and Worker Compatibility",[14,963,964],{},"Background workers (Celery, ARQ, Dramatiq) traditionally run in synchronous runtimes, creating an async\u002Fawait boundary mismatch when interacting with async drivers. Bridging this gap requires careful event loop management.",[14,966,967,970,971,974,975,978,979,981],{},[42,968,969],{},"Mitigating Blocking & Starvation:"," Never instantiate an async engine inside a synchronous worker thread and call ",[18,972,973],{},"await"," without a running loop. Instead, use ",[18,976,977],{},"asyncio.run()"," for isolated task execution, or configure the worker to run a dedicated event loop per process. Connection starvation occurs when workers exhaust the pool faster than the main application; isolate worker connection pools with separate ",[18,980,744],{}," configurations.",[14,983,984,985,30],{},"Production patterns for distributed queues are explored in ",[26,986,988],{"href":987},"\u002Fasync-engines-dialects-and-connection-pooling\u002Fchoosing-between-asyncpg-and-psycopg-async-drivers\u002Fusing-sqlalchemy-async-with-celery-task-workers\u002F","Using SQLAlchemy async with Celery Task Workers",[32,990,992],{"id":991},"psycopg3-async-implementation-deep-dive","psycopg3 Async Implementation Deep Dive",[14,994,995,996,998,999,1001],{},"Migrating from ",[18,997,98],{}," to ",[18,1000,58],{}," async adapters requires minimal refactoring due to SQLAlchemy's dialect abstraction, but unlocks modern PostgreSQL capabilities.",[14,1003,1004],{},[42,1005,1006],{},"Key Advantages:",[1008,1009,1010,1017,1023],"ul",{},[1011,1012,1013,1016],"li",{},[42,1014,1015],{},"Server-Side Cursors:"," Efficiently stream millions of rows without exhausting client memory.",[1011,1018,1019,1022],{},[42,1020,1021],{},"COPY FROM\u002FTO:"," High-speed bulk data transfer bypassing standard INSERT overhead.",[1011,1024,1025,1028,1029,400,1032,1035,1036,1039,1040,79,1043,1046],{},[42,1026,1027],{},"Advanced Isolation:"," Native support for ",[18,1030,1031],{},"READ COMMITTED",[18,1033,1034],{},"REPEATABLE READ",", and ",[18,1037,1038],{},"SERIALIZABLE"," with explicit ",[18,1041,1042],{},"begin()",[18,1044,1045],{},"commit()"," boundaries.",[14,1048,1049],{},"Step-by-step migration and configuration are documented in Using psycopg3 Async Driver with SQLAlchemy 2.0.",[14,1051,1052,1053,1055,1056,1059,1060,1063],{},"When using ",[18,1054,58],{},", explicitly configure ",[18,1057,1058],{},"statement_cache_size=0"," if you encounter plan cache invalidation issues with volatile query parameters, and leverage ",[18,1061,1062],{},"psycopg.types"," for custom JSON\u002FUUID adapters.",[32,1065,1067],{"id":1066},"driver-extensibility-and-custom-dialect-architecture","Driver Extensibility and Custom Dialect Architecture",[14,1069,1070],{},"While both drivers cover 95% of production use cases, specialized workloads (PostGIS, TimescaleDB, custom ENUMs, or proprietary extensions) may require dialect overrides.",[14,1072,1073,1076,1077,1080,1081,1084,1085,1088,1089,1092,1093,1096,1097,1100],{},[42,1074,1075],{},"Custom Type Compilers & Execution Contexts:"," SQLAlchemy 2.0 allows you to register custom ",[18,1078,1079],{},"TypeDecorator"," implementations that map Python objects to driver-specific wire formats. For unsupported PostgreSQL extensions, you can subclass ",[18,1082,1083],{},"PGDialect_asyncpg"," or ",[18,1086,1087],{},"PGDialect_psycopg"," to override ",[18,1090,1091],{},"do_execute()",", inject custom ",[18,1094,1095],{},"SET"," commands on connection checkout, or modify the SQL compiler's ",[18,1098,1099],{},"visit_column()"," logic.",[14,1102,1103],{},"Advanced extension techniques are covered in Writing Custom Dialects for Unsupported Databases.",[14,1105,1106],{},"Always benchmark custom dialect hooks against raw driver execution to ensure the abstraction layer doesn't introduce unacceptable latency.",[1108,1109],"hr",{},[32,1111,1113],{"id":1112},"production-pitfalls-to-avoid","Production Pitfalls to Avoid",[1115,1116,1117,1133,1153,1170,1184],"ol",{},[1011,1118,1119,1122,1123,1125,1126,1128,1129,1132],{},[42,1120,1121],{},"Mixing Sync and Async Drivers:"," Instantiating a synchronous ",[18,1124,98],{}," engine alongside an async ",[18,1127,44],{}," engine in the same event loop will cause ",[18,1130,1131],{},"RuntimeError: cannot schedule new futures after shutdown"," or silent blocking.",[1011,1134,1135,1138,1139,1141,1142,1145,1146,1149,1150,30],{},[42,1136,1137],{},"Improper Pool Sizing:"," Setting ",[18,1140,744],{}," too low causes ",[18,1143,1144],{},"QueuePool"," exhaustion under burst traffic; setting ",[18,1147,1148],{},"max_overflow"," too high overwhelms PostgreSQL's ",[18,1151,1152],{},"max_connections",[1011,1154,1155,1158,1159,1162,1163,1166,1167,30],{},[42,1156,1157],{},"Neglecting Session Closure:"," Failing to ",[18,1160,1161],{},"await session.close()"," or using bare ",[18,1164,1165],{},"yield session"," without context managers leaks connections, eventually triggering ",[18,1168,1169],{},"FATAL: too many connections for role",[1011,1171,1172,89,1175,75,1177,1180,1181,1183],{},[42,1173,1174],{},"Using Legacy psycopg2 Async Adapters:",[18,1176,98],{},[18,1178,1179],{},"psycopg2.extras.wait_select"," is a synchronous wrapper masquerading as async. It blocks the loop. Always use ",[18,1182,62],{}," (v3+).",[1011,1185,1186,89,1189,1191,1192,1195],{},[42,1187,1188],{},"Ignoring Statement Cache Limits:",[18,1190,44],{},"'s default 100-statement LRU cache can cause memory pressure in microservices with thousands of unique query shapes. Tune ",[18,1193,1194],{},"statement_cache_size"," or disable it for highly dynamic workloads.",[32,1197,1199],{"id":1198},"frequently-asked-questions","Frequently Asked Questions",[14,1201,1202,1205,1206,1208,1209,1211,1212,1214],{},[42,1203,1204],{},"Is asyncpg faster than psycopg3 for SQLAlchemy 2.0?","\nGenerally, yes. ",[18,1207,44],{},"'s native async implementation and binary protocol yield lower latency and higher throughput for parameterized, repetitive queries. ",[18,1210,58],{}," trades marginal raw speed for broader PostgreSQL feature parity, ",[18,1213,48],{}," stability, and easier migration paths from legacy codebases.",[14,1216,1217,1220,1221,998,1224,1227,1228,400,1231,1234,1235,1237],{},[42,1218,1219],{},"Can I switch from asyncpg to psycopg3 without rewriting queries?","\nYes. SQLAlchemy 2.0's dialect abstraction isolates query construction from driver execution. Changing the URL prefix from ",[18,1222,1223],{},"postgresql+asyncpg:\u002F\u002F",[18,1225,1226],{},"postgresql+psycopg:\u002F\u002F"," is typically sufficient. You may need to adjust ",[18,1229,1230],{},"pool_recycle",[18,1232,1233],{},"connect_args",", and disable driver-specific features like ",[18,1236,44],{},"'s automatic prepared statement caching if they conflict with your workload.",[14,1239,1240,1243,1244,1247,1248,1251,1252,1255,1256,749,1258,1261,1262,1265,1266,1269],{},[42,1241,1242],{},"How do I handle connection leaks in async SQLAlchemy?","\nAlways wrap sessions in ",[18,1245,1246],{},"async with"," blocks or custom async context managers. Implement explicit ",[18,1249,1250],{},"try\u002Fexcept"," blocks that guarantee ",[18,1253,1254],{},"await session.rollback()"," on failure. Configure ",[18,1257,1230],{},[18,1259,1260],{},"pool_timeout"," to automatically reclaim stale connections, and monitor ",[18,1263,1264],{},"pool.checkedin"," vs ",[18,1267,1268],{},"pool.checkedout"," metrics in production.",[14,1271,1272,1275,1277],{},[42,1273,1274],{},"Which driver is better for microservices with high concurrency?",[18,1276,44],{}," is typically preferred for cloud-native, high-concurrency microservices due to its lightweight architecture, efficient connection pooling, and minimal event loop overhead. Ensure your entire stack (ASGI server, HTTP client, cache layer) is fully async-native to realize the performance benefits.",[1279,1280,1281],"style",{},"html pre.shiki code .szBVR, html code.shiki .szBVR{--shiki-default:#D73A49;--shiki-dark:#F97583}html pre.shiki code .sVt8B, html code.shiki .sVt8B{--shiki-default:#24292E;--shiki-dark:#E1E4E8}html pre.shiki code .sScJk, html code.shiki .sScJk{--shiki-default:#6F42C1;--shiki-dark:#B392F0}html pre.shiki code .sj4cs, html code.shiki .sj4cs{--shiki-default:#005CC5;--shiki-dark:#79B8FF}html pre.shiki code .sZZnC, html code.shiki .sZZnC{--shiki-default:#032F62;--shiki-dark:#9ECBFF}html pre.shiki code .sJ8bj, html code.shiki .sJ8bj{--shiki-default:#6A737D;--shiki-dark:#6A737D}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .s4XuR, html code.shiki .s4XuR{--shiki-default:#E36209;--shiki-dark:#FFAB70}",{"title":145,"searchDepth":169,"depth":169,"links":1283},[1284,1285,1286,1287,1288,1289,1290,1291,1292],{"id":34,"depth":169,"text":35},{"id":106,"depth":169,"text":107},{"id":333,"depth":169,"text":334},{"id":764,"depth":169,"text":765},{"id":960,"depth":169,"text":961},{"id":991,"depth":169,"text":992},{"id":1066,"depth":169,"text":1067},{"id":1112,"depth":169,"text":1113},{"id":1198,"depth":169,"text":1199},"When architecting high-throughput, non-blocking Python services, selecting the correct PostgreSQL driver is a foundational infrastructure decision. The asyncpg vs psycopg async driver debate centers on execution models, protocol efficiency, and ecosystem compatibility. SQLAlchemy 2.0’s modern async API provides a unified abstraction layer, but understanding the underlying driver mechanics is critical for avoiding event loop starvation, optimizing connection lifecycle, and achieving predictable latency.","md",{},"\u002Fasync-engines-dialects-and-connection-pooling\u002Fchoosing-between-asyncpg-and-psycopg-async-drivers",{"title":5,"description":1293},"async-engines-dialects-and-connection-pooling\u002Fchoosing-between-asyncpg-and-psycopg-async-drivers\u002Findex","z0VzaYP4eEY7TYNze0DK1VPxCnox3nIa-CIaHwDpzTw",1778149144399]