Pyspark error inserting into mysql database table that exists
1
vote
0
answers
422
views
I am trying to insert into an existing mysql table using Pyspark JDBC connection however I get the following error(picture attached)
Can I get assistance on this error. The table exist in the MySql Database, I was successful in Inserting with a Dataframe when selecting data form another table , however i would like to Insert directly to the Mysql datadase table.
Below is the Pyspark code I am using to insert into the table:

import os
from pyspark.sql import SQLContext
from pyspark import SparkContext
from pyspark.sql import SparkSession
import logging
sc = SparkSession.builder.appName("TestConnection").enableHiveSupport().getOrCreate()
sqlContext = SQLContext(sc)
Provide your Spark-master node below
hostname = "localhost"
dbname = "dev_db"
username = "******"
password = "*******"
jdbc_url = "jdbc:mysql://{0}/{1}?user={2}&password={3}".format(hostname,dbname,username,password)
sqlContext.sql("insert into test values ('3','Names',123654)").write.format('jdbc').options(url=jdbc_url,driver='com.mysql.jdbc.Driver').mode(append).save()
logging.info("***********Write to MySql Completed***********")
Asked by IIShriyaII
(11 rep)
Jul 16, 2020, 04:09 PM
Last activity: Jul 16, 2020, 08:15 PM
Last activity: Jul 16, 2020, 08:15 PM