Alembic upgrade fails for multiple migration files #184
Description
Using Cloud Spanner emulator I am allowed to run my current models and get the tables created, although when using Alembic migrations in the same scenario I keep getting errors from Spanner, warning that it cannot update a primary key row - which I conclude is the num version of the file.
Environment details
Windows 11, using Cloud Spanner emulator.
Requirements.txt:
alembic==1.7.5 asgiref==3.4.1 black==21.9b0 cachetools==4.2.4 certifi==2021.10.8 charset-normalizer==2.0.9 click==8.0.1 colorama==0.4.4 coverage==6.1.1 fastapi==0.68.2 google-api-core==2.3.2 google-auth==2.3.3 google-cloud-core==2.2.1 google-cloud-spanner==3.12.0 googleapis-common-protos==1.54.0 greenlet==1.1.2 grpc-google-iam-v1==0.12.3 grpcio==1.43.0 grpcio-status==1.43.0 h11==0.12.0 idna==3.3 linecache2==1.0.0 Mako==1.1.6 MarkupSafe==2.0.1 mypy-extensions==0.4.3 packaging==21.3 pathspec==0.9.0 platformdirs==2.4.0 proto-plus==1.19.8 protobuf==3.19.1 psycopg==3.0.1 pyasn1==0.4.8 pyasn1-modules==0.2.8 pydantic==1.8.2 pyparsing==3.0.6 python-dotenv==0.19.0 regex==2021.10.23 requests==2.26.0 rsa==4.8 six==1.16.0 SQLAlchemy==1.3.23 sqlalchemy-spanner==1.0.0 sqlparse==0.4.2 starlette==0.14.2 tomli==1.2.2 traceback2==1.4.0 typing-extensions==3.10.0.2 unittest2==1.1.0 urllib3==1.26.7 uvicorn==0.15.0
Steps to reproduce
- Add more than one migration file
- Run
alembic upgrade head
Logs
alembic upgrade head >> log.log
INFO [alembic.runtime.migration] Context impl SpannerImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
INFO [alembic.runtime.migration] Running upgrade 223746b7a8ff -> 83237b012e14, create flavours table
Traceback (most recent call last):
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\google\api_core\grpc_helpers.py", line 66, in error_remapped_callable
return callable_(*args, **kwargs)
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\grpc_channel.py", line 946, in call
return _end_unary_response_blocking(state, call, False, None)
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\grpc_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.FAILED_PRECONDITION
details = "Schema change operation rejected because a concurrent schema change operation or read-write transaction is already in progress."
debug_error_string = "{"created":"@1640894679.713000000","description":"Error received from peer ipv4:127.0.0.1:9010","file":"src/core/lib/surface/call.cc","file_line":1075,"grpc_message":"Schema change operation rejected because a concurrent schema change operation or read-write transaction is already in progress.","grpc_status":9}"
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\google\cloud\spanner_dbapi\cursor.py", line 239, in execute
self.connection.run_prior_DDL_statements()
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\google\cloud\spanner_dbapi\connection.py", line 55, in wrapper
return function(connection, *args, **kwargs)
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\google\cloud\spanner_dbapi\connection.py", line 413, in run_prior_DDL_statements
return self.database.update_ddl(ddl_statements).result()
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\google\cloud\spanner_v1\database.py", line 447, in update_ddl
future = api.update_database_ddl(request=request, metadata=metadata)
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\google\cloud\spanner_admin_database_v1\services\database_admin\client.py", line 822, in update_database_ddl
response = rpc(request, retry=retry, timeout=timeout, metadata=metadata,)
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\google\api_core\gapic_v1\method.py", line 154, in call
return wrapped_func(*args, **kwargs)
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\google\api_core\retry.py", line 283, in retry_wrapped_func
return retry_target(
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\google\api_core\retry.py", line 190, in retry_target
return target()
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\google\api_core\grpc_helpers.py", line 68, in error_remapped_callable
raise exceptions.from_grpc_error(exc) from exc
google.api_core.exceptions.FailedPrecondition: 400 Schema change operation rejected because a concurrent schema change operation or read-write transaction is already in progress.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\google\cloud\sqlalchemy_spanner\sqlalchemy_spanner.py", line 975, in do_execute
cursor.execute(statement, parameters)
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\google\cloud\spanner_dbapi\cursor.py", line 69, in wrapper
return function(cursor, *args, **kwargs)
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\google\cloud\spanner_dbapi\cursor.py", line 278, in execute
raise IntegrityError(getattr(e, "details", e))
google.cloud.spanner_dbapi.exceptions.IntegrityError: []
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "c:\users\zappts\appdata\local\programs\python\python39\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "c:\users\zappts\appdata\local\programs\python\python39\lib\runpy.py", line 87, in run_code
exec(code, run_globals)
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\Scripts\alembic.exe_main.py", line 7, in
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\alembic\config.py", line 588, in main
CommandLine(prog=prog).main(argv=argv)
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\alembic\config.py", line 582, in main
self.run_cmd(cfg, options)
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\alembic\config.py", line 559, in run_cmd
fn(
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\alembic\command.py", line 320, in upgrade
script.run_env()
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\alembic\script\base.py", line 563, in run_env
util.load_python_file(self.dir, "env.py")
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\alembic\util\pyfiles.py", line 92, in load_python_file
module = load_module_py(module_id, path)
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\alembic\util\pyfiles.py", line 108, in load_module_py
spec.loader.exec_module(module) # type: ignore
File "", line 850, in exec_module
File "", line 228, in call_with_frames_removed
File "alembic\env.py", line 95, in
run_migrations_online()
File "alembic\env.py", line 89, in run_migrations_online
context.run_migrations()
File "", line 8, in run_migrations
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\alembic\runtime\environment.py", line 851, in run_migrations
self.get_context().run_migrations(**kw)
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\alembic\runtime\migration.py", line 627, in run_migrations
head_maintainer.update_to_step(step)
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\alembic\runtime\migration.py", line 851, in update_to_step
self.update_version(from, to)
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\alembic\runtime\migration.py", line 788, in _update_version
ret = self.context.impl._exec(
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\alembic\ddl\impl.py", line 197, in _exec
return conn.execute(construct, multiparams)
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1011, in execute
return meth(self, multiparams, params)
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\sqlalchemy\sql\elements.py", line 298, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1124, in _execute_clauseelement
ret = self._execute_context(
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1316, in _execute_context
self.handle_dbapi_exception(
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1510, in handle_dbapi_exception
util.raise(
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\sqlalchemy\util\compat.py", line 182, in raise
raise exception
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\sqlalchemy\engine\base.py", line 1276, in _execute_context
self.dialect.do_execute(
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\google\cloud\sqlalchemy_spanner\sqlalchemy_spanner.py", line 975, in do_execute
cursor.execute(statement, parameters)
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\google\cloud\spanner_dbapi\cursor.py", line 69, in wrapper
return function(cursor, *args, **kwargs)
File "C:\Users\Zappts\OneDrive\Área de Trabalho\microservices\github\plk-microservice-product\venv\lib\site-packages\google\cloud\spanner_dbapi\cursor.py", line 278, in execute
raise IntegrityError(getattr(e, "details", e))
sqlalchemy.exc.IntegrityError: (google.cloud.spanner_dbapi.exceptions.IntegrityError) []
[SQL: UPDATE alembic_version SET version_num='83237b012e14' WHERE alembic_version.version_num = '223746b7a8ff']
(Background on this error at: http://sqlalche.me/e/13/gkpj)