Data Source Objects and Libraries

This article will cover all of the data types that are returned by Zepl’s secure data source objects. In order to build a standard method for accessing any data source, Zepl has built data source connectors to support one or all of the four programming languages used by data scientists; Python, R, Scala, and SQL.

Zepl has created a simple, secure, and standard method for all users to access their data. Zepl’s data sources allow each user to securely store their user credentials through our encrypted keychain.

Each programming language supports different access methods to each of these data sources. We will walk through what objects are returned upon connecting to each data source. This will give you the best insight to take advantage of our features and troubleshoot any errors that may arise.


Snowflake

Configure

Note: If your Warehouse is not specified, the user will need to run the "USE WAREHOUSE " command in the language of their choice. I.e. if you are connecting through python, select warehouse in this way:

cur = z.getDatasource("<data source name>")
cur.execute("USE WAREHOUSE <warehouse>")

Read from Snowflake

Open Examples in Zepl

Language Code Object Returned Library
Python cur = z.getDatasource("<data source name>")

conn = z.getDatasource("<data source name>_con")
Snowflake Cursor Object

Snowflake Connection Object
Snowflake-connector-python - v2.0.3
R / SparkR conn <- z.getDatasource("<data source name>") DBIConnection object: Zepl function uses the dbConnect() function from the DPLYR library
Scala val df = z.getDatasource("<data source name>").asInstanceOf[org.apache.spark.sql.DataFrameReader] Spark SQL Dataframe: org.apache.spark.sql.DataFrameReader
SQL %datasource.<data source name> JDBC connection to Snowflake

Note: Spark (Scala and Pyspark) require 'STAGE' permissions on the Snowflake database or else this exception may be thrown: net.snowflake.client.jdbc.SnowflakeSQLException: SQL execution error: Creating stage on shared database 'SNOWFLAKE_SAMPLE_DATA' is not allowed

Write data to Snowflake

Open Examples in Zepl

Amazon S3

Configure

Read data from Amazon S3

Open Examples in Zepl

Language Code Object Returned Library
Python s3_bucket = z.getDatasource("<data source name>") Boto3 Bucket Object Boto3 - v1.10.9

Write data to Amazon S3

Open Examples in Zepl

Google BigQuery

Configure

Read data from BigQuery

Open Examples in Zepl

Language Code Object Returned Library
Python client = z.getDatasource("<data source name>") BigQuery Client Object google-cloud-bigquery - v1.21.0

Write data to BigQuery

Example coming soon!

MySQL

Configure

Read data from MySQL

Open Examples in Zepl

Language Code Object Returned Library
Python conn = z.getDatasource("<data source name>") mysql.connector.connect mysql-connector-python - v8.0.18
SQL %datasource.<data source name>

Write data to MySQL

Example coming soon!

PostgreSQL

Configure

Read data from PostgreSQL

Open Examples in Zepl

Language Code Object Returned Library
Python conn = z.getDatasource("<data source name>") psycopg2.connect psycopg2-binary - v2.8.4
SQL %datasource.<data source name>

Write data to PostgreSQL

Example coming soon!

Apache Cassandra

Configure

Read data from Apache Cassandra

Open Examples in Zepl

Language Code Object Returned Library
Python session = z.getDatasource("<data source name>") cluster.connect.session cassandra-driver - v3.20.0

Write data to Apache Cassandra

Example coming soon!

SAP HANA

Configure

Read data from SAP HANA

Example coming soon!

Language Code Object Returned Library
Python conn = z.getDatasource("<data source name>") dbapi.connect hdbcli - v2.4.167
Scala val df = z.getDatasource("<data source name>").asInstanceOf[org.apache.spark.sql.DataFrameReader] Spark SQL Dataframe: org.apache.spark.sql.DataFrameReader

Write data to SAP HANA

Example coming soon!

Alibaba MaxCompute

Configure

Read data from Alibaba MaxCompute

Example coming soon!

Language Code Object Returned Library
Python odps = z.getDatasource("<data source name>") ODPS Object pyodps - v0.8.4
Scala val df = z.getDatasource("<data source name>").asInstanceOf[org.apache.spark.sql.DataFrameReader] Spark SQL Dataframe: org.apache.spark.sql.DataFrameReader odps-jdbc-3.1.0.jar

Read data to Alibaba MaxCompute

Example coming soon!