我开发了一个 Python 脚本，可以将数据导入到我的 Postgres DB，它在我的本地机器上运行良好，但是当我将其传输到服务器时，它失败了：
Traceback (most recent call last):
File "./importer.py", line 1, in <module>
import psycopg2
ModuleNotFoundError: No module named 'psycopg2'
下面是python脚本：
import psycopg2
import glob, os
import zipfile
import ntpath
## db connection string and objects
t_host = "localhost"
t_dbname = "*****"
t_username = "******"
t_password = "******"
t_port = 5432
t_schema = "******"
t_stageTable = "******"
## working directories
inputDir = "./incoming"
outputDir = "./processed"
## functions definition
def clearDirectory(dir):
for csvFile in glob.glob(dir+"/*.csv"):
os.remove(csvFile)
def stripQuotes(str):
return str.strip('"')
def uploadFile(file):
db_conn = psycopg2.connect(host=t_host, port=t_port, dbname=t_dbname, user=t_username, password=t_password)
db_cursor = db_conn.cursor()
f_contents = open(file, 'r')
for row in f_contents:
record = row.rstrip().split(",")
record.append('"'+ntpath.basename(file)+'"')
insert_query = "INSERT INTO ...")
#commit after the whole file is loaded
db_conn.commit()
#close connection
db_cursor.close()
db_conn.close()
def processFiles(dirIn, dirOut):
for file in glob.glob(dirIn+"/*.zip"):
#extract files
with zipfile.ZipFile(file, 'r') as zip_ref:
zip_ref.extractall(dirOut)
for file in glob.glob(dirOut+"/*.csv"):
uploadFile(file)
#move .zip files to processed folder
for file in glob.glob(dirIn+"/*.zip"):
os.replace(file, dirOut+"/"+ntpath.basename(file))
###################################
## actual processing
###################################
#clean old files first (just to be sure)
clearDirectory(outputDir)
#loop through zip files in the incoming folder
processFiles(inputDir, outputDir)
#clean again
clearDirectory(outputDir)
###################################
psycopg2我尝试使用 pip3安装缺少的库
sudo pip3 install psycopg2
但我收到了这个奇怪的错误：
Collecting psycopg2
Using cached psycopg2-2.8.6.tar.gz (383 kB)
Building wheels for collected packages: psycopg2
Building wheel for psycopg2 (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-80qwhbn5/psycopg2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-80qwhbn5/psycopg2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-uwxpzq13
cwd: /tmp/pip-install-80qwhbn5/psycopg2/
Complete output (40 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.8
creating build/lib.linux-x86_64-3.8/psycopg2
copying lib/_json.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/extensions.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/_ipaddress.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/extras.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/errorcodes.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/compat.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/sql.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/_range.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/tz.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/_lru_cache.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/errors.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/__init__.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/pool.py -> build/lib.linux-x86_64-3.8/psycopg2
running build_ext
building 'psycopg2._psycopg' extension
creating build/temp.linux-x86_64-3.8
creating build/temp.linux-x86_64-3.8/psycopg
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPSYCOPG_VERSION=2.8.6 (dt dec pq3 ext lo64) -DPG_VERSION_NUM=120006 -DHAVE_LO64=1 -I/usr/include/python3.8 -I. -I/usr/include/postgresql -I/usr/include/postgresql/12/server -c psycopg/psycopgmodule.c -o build/temp.linux-x86_64-3.8/psycopg/psycopgmodule.o -Wdeclaration-after-statement
In file included from psycopg/psycopgmodule.c:28:
./psycopg/psycopg.h:36:10: fatal error: libpq-fe.h: No such file or directory
36 | #include <libpq-fe.h>
| ^~~~~~~~~~~~
compilation terminated.
It appears you are missing some prerequisite to build the package from source.
You may install a binary package by installing 'psycopg2-binary' from PyPI.
If you want to install psycopg2 from source, please install the packages
required for the build and try again.
For further information please check the 'doc/src/install.rst' file (also at
<https://www.psycopg.org/docs/install.html>).
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for psycopg2
Running setup.py clean for psycopg2
Failed to build psycopg2
Installing collected packages: psycopg2
Running setup.py install for psycopg2 ... error
ERROR: Command errored out with exit status 1:
command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-80qwhbn5/psycopg2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-80qwhbn5/psycopg2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-4g11q7zi/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.8/psycopg2
cwd: /tmp/pip-install-80qwhbn5/psycopg2/
Complete output (40 lines):
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.8
creating build/lib.linux-x86_64-3.8/psycopg2
copying lib/_json.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/extensions.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/_ipaddress.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/extras.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/errorcodes.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/compat.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/sql.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/_range.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/tz.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/_lru_cache.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/errors.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/__init__.py -> build/lib.linux-x86_64-3.8/psycopg2
copying lib/pool.py -> build/lib.linux-x86_64-3.8/psycopg2
running build_ext
building 'psycopg2._psycopg' extension
creating build/temp.linux-x86_64-3.8
creating build/temp.linux-x86_64-3.8/psycopg
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPSYCOPG_VERSION=2.8.6 (dt dec pq3 ext lo64) -DPG_VERSION_NUM=120006 -DHAVE_LO64=1 -I/usr/include/python3.8 -I. -I/usr/include/postgresql -I/usr/include/postgresql/12/server -c psycopg/psycopgmodule.c -o build/temp.linux-x86_64-3.8/psycopg/psycopgmodule.o -Wdeclaration-after-statement
In file included from psycopg/psycopgmodule.c:28:
./psycopg/psycopg.h:36:10: fatal error: libpq-fe.h: No such file or directory
36 | #include <libpq-fe.h>
| ^~~~~~~~~~~~
compilation terminated.
It appears you are missing some prerequisite to build the package from source.
You may install a binary package by installing 'psycopg2-binary' from PyPI.
If you want to install psycopg2 from source, please install the packages
required for the build and try again.
For further information please check the 'doc/src/install.rst' file (also at
<https://www.psycopg.org/docs/install.html>).
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-80qwhbn5/psycopg2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-80qwhbn5/psycopg2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-4g11q7zi/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.8/psycopg2 Check the logs for full command output.
以前从未见过这种情况。知道发生了什么吗？
我甚至尝试安装 psycopg2-binary ...成功了...但是 py 脚本仍然出现相同的错误->没有模块...
编辑：@西蒙·奥尔德里奇
sudo apt install libpg-dev最终结果为：
Reading package lists... Done
Building dependency tree
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:
The following packages have unmet dependencies:
libpq-dev : Depends: libpq5 (= 12.6-0ubuntu0.20.04.1) but 13.2-1.pgdg20.04+1 is to be installed
E: Unable to correct problems, you have held broken packages.
当我尝试删除 libpq5 时
sudo apt purge libpq5
它说：
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
libapache2-mod-wsgi-py3 libboost-filesystem1.71.0 libboost-thread1.71.0 libllvm10
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
libpq5* pgadmin4* pgadmin4-desktop* pgadmin4-server* pgadmin4-web* pgagent* postgresql* postgresql-12* postgresql-client-12*
postgresql-contrib*
0 upgraded, 0 newly installed, 10 to remove and 0 not upgraded.
我不希望删除 pgadmin :(
答案1
听起来你缺少了libpq-dev包含你需要的标题的包。