Python连接数据库
Python连接数据库,有多种方式,本文介绍三种方式,分别为通过SparkSession、jaydebeapi和jInsecureClient方式 连接数据库。
方式一:通过SparkSession方式连接
from pyspark.sql import SparkSession
import pandas as pd
spark = SparkSession.builder
.appName('myAppName')
.getOrCreate()
spark.conf.set("spark.sql.execution.arrow.enabled","false")
print(spark.sparkContext.applicationId)
show_tables_1 ="""
SELECT * FROM table
where stat_date between '2024-02-01' and '2024-04-03'
"""
df1 = spark.sql(show_tables_1).toPandas()
方式二:通过jaydebeapi方式
import jaydebeapi
import pandas as pd
import os
def downloaddata(sqlstr):
url='jdbc:presto:/hive'
user='user'
dirver='io.prestosql.jdbc.PrestoDriver'
current_dir = os.path.dirname(os.path.abspath(__file__))
jarFile=current_dir+'/presto-jdbc-306.jar'
conn=jaydebeapi.connect(dirver,url,[user,''],jarFile)
curs = conn.cursor()
curs.execute(sqlstr)
result=curs.fetchall()
if len(result)==0:
print("presto查询无结果!")
else :
dt = pd.DataFrame(result)
print(dt.head(10))
return dt
方式三:通过jInsecureClient方式
try:
client = InsecureClient(url="http://nn2.ns12.vip.bigdata",user="admin")
print('r1',client.list('/'))
except HdfsError as e:
if "Operation category READ is not supported in state standby" in str(e):
client = InsecureClient(url="http://nn1.ns12.vip.bigdata",user="admin")
print('r2',client.list('/'))
else:
print("其他 HdfsError 异常:", e)
关注公众号「水沐教育科技」,在手机上阅读所有教程,随时随地都能学习。内含一款搜索神器,免费下载全网书籍和视频。

微信扫码关注公众号