Context Navigation

Modify ↓

#13605 closed defect (fixed)

2 test failures with Python 3.11 on Windows (part of #13402)

Reported by:	Jun Omae	Owned by:	Jun Omae
Priority:	normal	Milestone:	1.6
Component:	database backend	Version:
Severity:	normal	Keywords:
Cc:		Branch:
Release Notes:
API Changes:
Internal Changes:

Description

I got 2 failures with Python 3.11 on Windows.

Python: /c/Users/runneradmin/AppData/Local/venv/Scripts/python 

  Package        Version
  -----------------------------------------------------------------------------------------------
  Python       : 3.11.1 (tags/v3.11.1:a7a450f, Dec  6 2022, 19:58:39) [MSC v.1934 64 bit (AMD64)]
  Setuptools   : 65.6.3
  Pip          : 22.3.1
  Wheel        : not installed
  Jinja2       : 3.1.2
  Babel        : not installed
  sqlite3      : 2.6.0 (3.39.4)
  PySqlite3    : not installed
  PyMySQL      : not installed
  Psycopg2     : not installed
  SVN bindings : not installed
  Mercurial    : not installed
  Pygments     : not installed
  Textile      : not installed
  Pytz         : not installed
  Docutils     : not installed
  Selenium     : 4.7.2
  PyTidyLib    : 0.3.2 (5.9.14 C:\Users\runneradmin\AppData\Local\venv\Scripts\tidy.dll)
  LXML         : not installed
  coverage     : not installed

Variables:
  PATH=/mingw64/bin:/usr/bin:/c/Users/runneradmin/bin:/c/Users/runneradmin/AppData/Local/venv/Scripts:/c/Users/runneradmin/AppData/Local/venv/Scripts/Scripts:/c/Users/runneradmin/AppData/Local/venv/Scripts:/c/Program Files/PowerShell/7:/c/Users/runneradmin/AppData/Roaming/Python/Python311/Scripts:/c/hostedtoolcache/windows/Python/3.11.1/x64/Scripts:/c/hostedtoolcache/windows/Python/3.11.1/x64:/c/Program Files/MongoDB/Server/5.0/bin:/c/aliyun-cli:/c/vcpkg:/c/Program Files (x86)/NSIS:/c/tools/zstd:/c/Program Files/Mercurial:/c/hostedtoolcache/windows/stack/2.9.1/x64:/c/cabal/bin:/c/ghcup/bin:/c/tools/ghc-9.4.2/bin:/c/Program Files/dotnet:/c/mysql/bin:/c/Program Files/R/R-4.2.2/bin/x64:/c/SeleniumWebDrivers/GeckoDriver:/c/Program Files (x86)/sbt/bin:/c/Program Files (x86)/GitHub CLI:/bin:/c/Program Files (x86)/pipx_bin:/c/npm/prefix:/c/hostedtoolcache/windows/go/1.17.13/x64/bin:/c/hostedtoolcache/windows/Python/3.9.13/x64/Scripts:/c/hostedtoolcache/windows/Python/3.9.13/x64:/c/hostedtoolcache/windows/Ruby/3.0.5/x64/bin:/c/tools/kotlinc/bin:/c/hostedtoolcache/windows/Java_Temurin-Hotspot_jdk/8.0.352-8/x64/bin:/c/Program Files/ImageMagick-7.1.0-Q16-HDRI:/c/Program Files (x86)/Microsoft SDKs/Azure/CLI2/wbin:/c/ProgramData/kind:/c/Program Files/Microsoft/jdk-11.0.12.7-hotspot/bin:/c/Windows/system32:/c/Windows:/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPowerShell/v1.0:/c/Windows/System32/OpenSSH:/c/Program Files/dotnet:/c/ProgramData/Chocolatey/bin:/c/Program Files/PowerShell/7:/c/Program Files/Microsoft/Web Platform Installer:/c/Program Files/Microsoft SQL Server/Client SDK/ODBC/170/Tools/Binn:/c/Program Files/Microsoft SQL Server/150/Tools/Binn:/c/Program Files/OpenSSL/bin:/c/Strawberry/c/bin:/c/Strawberry/perl/site/bin:/c/Strawberry/perl/bin:/c/ProgramData/chocolatey/lib/pulumi/tools/Pulumi/bin:/c/Program Files/TortoiseSVN/bin:/c/Program Files/CMake/bin:/c/ProgramData/chocolatey/lib/maven/apache-maven-3.8.6/bin:/c/Program Files/Microsoft Service Fabric/bin/Fabric/Fabric.Code:/c/Program Files/Microsoft SDKs/Service Fabric/Tools/ServiceFabricLocalClusterManager:/c/Program Files/nodejs:/cmd:/mingw64/bin:/usr/bin:/c/Program Files/GitHub CLI:/c/tools/php:/c/Program Files (x86)/sbt/bin:/c/SeleniumWebDrivers/ChromeDriver:/c/SeleniumWebDrivers/EdgeDriver:/c/Program Files/Amazon/AWSCLIV2:/c/Program Files/Amazon/SessionManagerPlugin/bin:/c/Program Files/Amazon/AWSSAMCLI/bin:/c/Program Files/Microsoft SQL Server/130/Tools/Binn:/c/Program Files/LLVM/bin:/c/Users/runneradmin/.dotnet/tools:/c/Users/runneradmin/.cargo/bin:/c/Users/runneradmin/AppData/Local/Microsoft/WindowsApps
  PYTHONPATH=.
  TRAC_TEST_DB_URI=
  server-options= -p 8000  -r -e 

External dependencies:
  Git version: git version 2.39.0.windows.1
  Subversion version: 1.14.2

...

======================================================================
ERROR: test_database_version (trac.tests.env.EnvironmentWithoutDataTestCase.test_database_version)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\hostedtoolcache\windows\Python\3.11.1\x64\Lib\shutil.py", line 620, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'c:\\users\\runneradmin\\appdata\\local\\temp\\trac-testdir-uqx7slhr\\db\\trac.db'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\a\trac\trac\trac\tests\env.py", line 47, in tearDown
    rmtree(self.env.path)
  File "D:\a\trac\trac\trac\test.py", line 549, in rmtree
    shutil.rmtree(path, onerror=onerror)
  File "C:\hostedtoolcache\windows\Python\3.11.1\x64\Lib\shutil.py", line 759, in rmtree
    return _rmtree_unsafe(path, onerror)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\hostedtoolcache\windows\Python\3.11.1\x64\Lib\shutil.py", line 617, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\hostedtoolcache\windows\Python\3.11.1\x64\Lib\shutil.py", line 622, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "D:\a\trac\trac\trac\test.py", line 544, in onerror
    onerror(function, path, excinfo, retry + 1)
  File "D:\a\trac\trac\trac\test.py", line 544, in onerror
    onerror(function, path, excinfo, retry + 1)
  File "D:\a\trac\trac\trac\test.py", line 544, in onerror
    onerror(function, path, excinfo, retry + 1)
  [Previous line repeated 7 more times]
  File "D:\a\trac\trac\trac\test.py", line 536, in onerror
    function(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'c:\\users\\runneradmin\\appdata\\local\\temp\\trac-testdir-uqx7slhr\\db\\trac.db'

======================================================================
ERROR: test_invalid_log_type_raises_exception (trac.tests.env.EnvironmentTestCase.test_invalid_log_type_raises_exception)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\hostedtoolcache\windows\Python\3.11.1\x64\Lib\shutil.py", line 620, in _rmtree_unsafe
    os.unlink(fullname)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'c:\\users\\runneradmin\\appdata\\local\\temp\\trac-testdir-1imzguf5\\db\\trac.db'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\a\trac\trac\trac\tests\env.py", line 106, in tearDown
    rmtree(self.env.path)
  File "D:\a\trac\trac\trac\test.py", line 549, in rmtree
    shutil.rmtree(path, onerror=onerror)
  File "C:\hostedtoolcache\windows\Python\3.11.1\x64\Lib\shutil.py", line 759, in rmtree
    return _rmtree_unsafe(path, onerror)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\hostedtoolcache\windows\Python\3.11.1\x64\Lib\shutil.py", line 617, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\hostedtoolcache\windows\Python\3.11.1\x64\Lib\shutil.py", line 622, in _rmtree_unsafe
    onerror(os.unlink, fullname, sys.exc_info())
  File "D:\a\trac\trac\trac\test.py", line 544, in onerror
    onerror(function, path, excinfo, retry + 1)
  File "D:\a\trac\trac\trac\test.py", line 544, in onerror
    onerror(function, path, excinfo, retry + 1)
  File "D:\a\trac\trac\trac\test.py", line 544, in onerror
    onerror(function, path, excinfo, retry + 1)
  [Previous line repeated 7 more times]
  File "D:\a\trac\trac\trac\test.py", line 536, in onerror
    function(path)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'c:\\users\\runneradmin\\appdata\\local\\temp\\trac-testdir-1imzguf5\\db\\trac.db'

----------------------------------------------------------------------
Ran 2465 tests in 261.398s

FAILED (errors=2, skipped=9)

Attachments (0)

Change History (3)

comment:1 by Jun Omae, 18 months ago

Owner:	set to Jun Omae
Status:	new → assigned

I noticed that, in Python 3.11, removing SQLite database file fails if I destroy a sqlite3.Connection instance without calling close. In Python 3.10, the removing is successful.

Python 3.11.3 (tags/v3.11.3:f3909b8, Apr  4 2023, 23:49:59) [MSC v.1934 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3, os, gc
>>> conn = sqlite3.connect('C:/usr/tmp/test.db')
>>> os.unlink('C:/usr/tmp/test.db')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:/usr/tmp/test.db'
>>> del conn
>>> os.unlink('C:/usr/tmp/test.db')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:/usr/tmp/test.db'
>>> gc.collect()
327
>>> os.unlink('C:/usr/tmp/test.db')
>>>
>>>
>>> conn = sqlite3.connect('C:/usr/tmp/test.db')
>>> os.unlink('C:/usr/tmp/test.db')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:/usr/tmp/test.db'
>>> conn.close()
>>> os.unlink('C:/usr/tmp/test.db')
>>>

Python 3.10.0 (tags/v3.10.0:b494f59, Oct  4 2021, 19:00:18) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3, os, gc
>>> conn = sqlite3.connect('C:/usr/tmp/test.db')
>>> os.unlink('C:/usr/tmp/test.db')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:/usr/tmp/test.db'
>>> del conn
>>> os.unlink('C:/usr/tmp/test.db')
>>>

Proposed changes:

trac/db/sqlite_backend.py

diff --git a/trac/db/sqlite_backend.py b/trac/db/sqlite_backend.py
index aa3194dd3..dc0bf4ca9 100644

                class SQLiteConnector(Component):
             # this direct connect will create the database if needed
             cnx = sqlite.connect(path, isolation_level=None,
                                  timeout=int(params.get('timeout', 10000)))
+            with closing(cnx.cursor()) as cursor:
+                _set_journal_mode(cursor, params.get('journal_mode'))
+                set_synchronous(cursor, params.get('synchronous'))
+                insert_schema(cursor, schema)
+            cnx.isolation_level = 'DEFERRED'
+            try:
+                with closing(cnx.cursor()) as cursor:
+                    _set_journal_mode(cursor, params.get('journal_mode'))
+                    set_synchronous(cursor, params.get('synchronous'))
+                    insert_schema(cursor, schema)
+                cnx.isolation_level = 'DEFERRED'
+                cnx.commit()
+            finally:
+                cnx.close()
         else:
             cnx = self.get_connection(path, log, params)
             with closing(cnx.cursor()) as cursor:
                 insert_schema(cursor, schema)
         cnx.commit()
+            cnx.commit()
     def destroy_db(self, path, log=None, params={}):
         if path != ':memory:':

comment:2 by Jun Omae, 18 months ago

Resolution:	→ fixed
Status:	assigned → closed

Fixed in [17704].

comment:3 by Jun Omae, 18 months ago

Python 3.10 on Windows in build workflow is still used…. Replaced it with Python 3.11 in [17705].

Modify Ticket

Change Properties

Summary:
Description:	I got 2 failures with Python 3.11 on Windows. {{{ Python: /c/Users/runneradmin/AppData/Local/venv/Scripts/python Package Version ----------------------------------------------------------------------------------------------- Python : 3.11.1 (tags/v3.11.1:a7a450f, Dec 6 2022, 19:58:39) [MSC v.1934 64 bit (AMD64)] Setuptools : 65.6.3 Pip : 22.3.1 Wheel : not installed Jinja2 : 3.1.2 Babel : not installed sqlite3 : 2.6.0 (3.39.4) PySqlite3 : not installed PyMySQL : not installed Psycopg2 : not installed SVN bindings : not installed Mercurial : not installed Pygments : not installed Textile : not installed Pytz : not installed Docutils : not installed Selenium : 4.7.2 PyTidyLib : 0.3.2 (5.9.14 C:\Users\runneradmin\AppData\Local\venv\Scripts\tidy.dll) LXML : not installed coverage : not installed Variables: PATH=/mingw64/bin:/usr/bin:/c/Users/runneradmin/bin:/c/Users/runneradmin/AppData/Local/venv/Scripts:/c/Users/runneradmin/AppData/Local/venv/Scripts/Scripts:/c/Users/runneradmin/AppData/Local/venv/Scripts:/c/Program Files/PowerShell/7:/c/Users/runneradmin/AppData/Roaming/Python/Python311/Scripts:/c/hostedtoolcache/windows/Python/3.11.1/x64/Scripts:/c/hostedtoolcache/windows/Python/3.11.1/x64:/c/Program Files/MongoDB/Server/5.0/bin:/c/aliyun-cli:/c/vcpkg:/c/Program Files (x86)/NSIS:/c/tools/zstd:/c/Program Files/Mercurial:/c/hostedtoolcache/windows/stack/2.9.1/x64:/c/cabal/bin:/c/ghcup/bin:/c/tools/ghc-9.4.2/bin:/c/Program Files/dotnet:/c/mysql/bin:/c/Program Files/R/R-4.2.2/bin/x64:/c/SeleniumWebDrivers/GeckoDriver:/c/Program Files (x86)/sbt/bin:/c/Program Files (x86)/GitHub CLI:/bin:/c/Program Files (x86)/pipx_bin:/c/npm/prefix:/c/hostedtoolcache/windows/go/1.17.13/x64/bin:/c/hostedtoolcache/windows/Python/3.9.13/x64/Scripts:/c/hostedtoolcache/windows/Python/3.9.13/x64:/c/hostedtoolcache/windows/Ruby/3.0.5/x64/bin:/c/tools/kotlinc/bin:/c/hostedtoolcache/windows/Java_Temurin-Hotspot_jdk/8.0.352-8/x64/bin:/c/Program Files/ImageMagick-7.1.0-Q16-HDRI:/c/Program Files (x86)/Microsoft SDKs/Azure/CLI2/wbin:/c/ProgramData/kind:/c/Program Files/Microsoft/jdk-11.0.12.7-hotspot/bin:/c/Windows/system32:/c/Windows:/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPowerShell/v1.0:/c/Windows/System32/OpenSSH:/c/Program Files/dotnet:/c/ProgramData/Chocolatey/bin:/c/Program Files/PowerShell/7:/c/Program Files/Microsoft/Web Platform Installer:/c/Program Files/Microsoft SQL Server/Client SDK/ODBC/170/Tools/Binn:/c/Program Files/Microsoft SQL Server/150/Tools/Binn:/c/Program Files/OpenSSL/bin:/c/Strawberry/c/bin:/c/Strawberry/perl/site/bin:/c/Strawberry/perl/bin:/c/ProgramData/chocolatey/lib/pulumi/tools/Pulumi/bin:/c/Program Files/TortoiseSVN/bin:/c/Program Files/CMake/bin:/c/ProgramData/chocolatey/lib/maven/apache-maven-3.8.6/bin:/c/Program Files/Microsoft Service Fabric/bin/Fabric/Fabric.Code:/c/Program Files/Microsoft SDKs/Service Fabric/Tools/ServiceFabricLocalClusterManager:/c/Program Files/nodejs:/cmd:/mingw64/bin:/usr/bin:/c/Program Files/GitHub CLI:/c/tools/php:/c/Program Files (x86)/sbt/bin:/c/SeleniumWebDrivers/ChromeDriver:/c/SeleniumWebDrivers/EdgeDriver:/c/Program Files/Amazon/AWSCLIV2:/c/Program Files/Amazon/SessionManagerPlugin/bin:/c/Program Files/Amazon/AWSSAMCLI/bin:/c/Program Files/Microsoft SQL Server/130/Tools/Binn:/c/Program Files/LLVM/bin:/c/Users/runneradmin/.dotnet/tools:/c/Users/runneradmin/.cargo/bin:/c/Users/runneradmin/AppData/Local/Microsoft/WindowsApps PYTHONPATH=. TRAC_TEST_DB_URI= server-options= -p 8000 -r -e External dependencies: Git version: git version 2.39.0.windows.1 Subversion version: 1.14.2 ... ====================================================================== ERROR: test_database_version (trac.tests.env.EnvironmentWithoutDataTestCase.test_database_version) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\hostedtoolcache\windows\Python\3.11.1\x64\Lib\shutil.py", line 620, in _rmtree_unsafe os.unlink(fullname) PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'c:\\users\\runneradmin\\appdata\\local\\temp\\trac-testdir-uqx7slhr\\db\\trac.db' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "D:\a\trac\trac\trac\tests\env.py", line 47, in tearDown rmtree(self.env.path) File "D:\a\trac\trac\trac\test.py", line 549, in rmtree shutil.rmtree(path, onerror=onerror) File "C:\hostedtoolcache\windows\Python\3.11.1\x64\Lib\shutil.py", line 759, in rmtree return _rmtree_unsafe(path, onerror) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\hostedtoolcache\windows\Python\3.11.1\x64\Lib\shutil.py", line 617, in _rmtree_unsafe _rmtree_unsafe(fullname, onerror) File "C:\hostedtoolcache\windows\Python\3.11.1\x64\Lib\shutil.py", line 622, in _rmtree_unsafe onerror(os.unlink, fullname, sys.exc_info()) File "D:\a\trac\trac\trac\test.py", line 544, in onerror onerror(function, path, excinfo, retry + 1) File "D:\a\trac\trac\trac\test.py", line 544, in onerror onerror(function, path, excinfo, retry + 1) File "D:\a\trac\trac\trac\test.py", line 544, in onerror onerror(function, path, excinfo, retry + 1) [Previous line repeated 7 more times] File "D:\a\trac\trac\trac\test.py", line 536, in onerror function(path) PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'c:\\users\\runneradmin\\appdata\\local\\temp\\trac-testdir-uqx7slhr\\db\\trac.db' ====================================================================== ERROR: test_invalid_log_type_raises_exception (trac.tests.env.EnvironmentTestCase.test_invalid_log_type_raises_exception) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\hostedtoolcache\windows\Python\3.11.1\x64\Lib\shutil.py", line 620, in _rmtree_unsafe os.unlink(fullname) PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'c:\\users\\runneradmin\\appdata\\local\\temp\\trac-testdir-1imzguf5\\db\\trac.db' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "D:\a\trac\trac\trac\tests\env.py", line 106, in tearDown rmtree(self.env.path) File "D:\a\trac\trac\trac\test.py", line 549, in rmtree shutil.rmtree(path, onerror=onerror) File "C:\hostedtoolcache\windows\Python\3.11.1\x64\Lib\shutil.py", line 759, in rmtree return _rmtree_unsafe(path, onerror) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\hostedtoolcache\windows\Python\3.11.1\x64\Lib\shutil.py", line 617, in _rmtree_unsafe _rmtree_unsafe(fullname, onerror) File "C:\hostedtoolcache\windows\Python\3.11.1\x64\Lib\shutil.py", line 622, in _rmtree_unsafe onerror(os.unlink, fullname, sys.exc_info()) File "D:\a\trac\trac\trac\test.py", line 544, in onerror onerror(function, path, excinfo, retry + 1) File "D:\a\trac\trac\trac\test.py", line 544, in onerror onerror(function, path, excinfo, retry + 1) File "D:\a\trac\trac\trac\test.py", line 544, in onerror onerror(function, path, excinfo, retry + 1) [Previous line repeated 7 more times] File "D:\a\trac\trac\trac\test.py", line 536, in onerror function(path) PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'c:\\users\\runneradmin\\appdata\\local\\temp\\trac-testdir-1imzguf5\\db\\trac.db' ---------------------------------------------------------------------- Ran 2465 tests in 261.398s FAILED (errors=2, skipped=9) }}} You may use WikiFormatting here.
Type:		Priority:
Milestone:		Component:
Version:		Severity:
Keywords:		Cc:	Set your email in Preferences
Branch:
Release Notes:
API Changes:
Internal Changes:

Action

leave as closed The owner will remain Jun Omae.

reopen The resolution will be deleted. Next status will be 'reopened'.

change ownership to The owner will be changed from Jun Omae to the specified user.

Add Comment

Your email or username:

E-mail address and name can be saved in the Preferences .

You may use WikiFormatting here.

Attachments ↑ Description ↑

Note: See TracTickets for help on using tickets.

Download in other formats: