Setup Presto SQL Development Environment

Presto SQL a massively parallel processing big-data engine grasps the attention of many big-data developers. This article is for those who like to set up a development environment for the Presto SQL community edition. The below-mentioned steps are applicable for any Presto variations including Presto DB with class names and file names replaced by equivalent names.

Requirements:

Setup Presto SQL Development Environment



Step 01:
Clone the master branch from the official Git repository
git clone https://github.com/prestosql/presto.git

Step 02:
Change the directory to the presto folder and build the project ignoring tests (Presto tests require docker and may take hours to complete)
cd presto
mvn clean install -DskipTests

Wait until the build to complete successfully.

Setup Presto SQL Development Environment

Running Presto from IntelliJ requires the local maven repository to be available at ~/.m2/repository. If your local repository is somewhere else, create a soft link folder at ~/.m2/repository pointing to the original location


Step 03:
Open IntelliJ IDEA and import the presto folder as a project.

Setup Presto SQL Development Environment

Step 04:
Once the project is imported, it should look like this:

Setup Presto SQL Development Environment

Step 05 [Optional]:
If you are going to test memory-intensive queries while running Presto from IntelliJ, it is advised to increase the JVM maximum heap size. Processes started from IntelliJ IDEA are limited by the max heap size of IntelliJ IDEA. Therefore, increase the heap size of IntelliJ IDEA from HelpEdit Custom VM Options...

-Xms2G
-Xmx15G
-XX:ReservedCodeCacheSize=240m
-XX:+UseConcMarkSweepGC
-XX:SoftRefLRUPolicyMSPerMB=50
-ea
-XX:CICompilerCount=2
-Dsun.io.useCanonPrefixCache=false
-Djava.net.preferIPv4Stack=true
-Djdk.http.auth.tunneling.disabledSchemes=""
-XX:+HeapDumpOnOutOfMemoryError
-XX:-OmitStackTraceInFastThrow
-Djdk.attach.allowAttachSelf
-Dawt.useSystemAAFontSettings=lcd
-Dsun.java2d.renderer=sun.java2d.marlin.MarlinRenderingEngine
-Dsun.tools.attach.tmp.only=true

Modify the Xms and Xmx properties according to the available memory and restart IntelliJ IDEA to apply the change.

Step 06:
Open the Project Structure from the File menu (Shotcut is Ctrl + Alt + Shift + S) and make sure that the Java 8 is  selected as the Project SDK as shown below. For the latest Presto source (after March 2020), choose Java 11.


Setup Presto SQL Development Environment

If you are using Java 11 to build, Presto, disable the Use --rebase option for corss-compilation (Java 9 and later) from Settings to avoid an issue in resolving sun.misc package.

Setup Presto SQL Development Environment

Step 07:
From the Build menu, select Build Project (Ctrl + F9) to build the project from IntelliJ IDEA.


Step 08:
Delete all unnecessary catalog properties from the presto-main/etc/catalog folder. For a hello world execution, tpch.properties and tpcds.properties are good enough. Catalog property with invalid configuration will prevent Presto server from starting up.

Setup Presto SQL Development Environment

Step 09:
Click on Add Configuration... option

Setup Presto SQL Development Environment

Step 10:
Select the Application template from the left side panel and enter the following details:

Setup Presto SQL Development Environment

Main class:
io.prestosql.server.PrestoServer

VM options:
-ea -XX:+UseG1GC -XX:G1HeapRegionSize=32M -XX:+UseGCOverheadLimit -XX:+ExplicitGCInvokesConcurrent -Xmx2G -Dconfig=etc/config.properties -Dlog.levels-file=etc/log.properties -Djdk.attach.allowAttachSelf=true

Increase the max heap memory in this argument if you are planning to execute memory intensive queries. The max heap size of Presto should be less than the max heap size of IntelliJ IDEA set in Setp 05.

Working directory:
$MODULE_DIR$

Use classpath of module:
presto-main

Setup Presto SQL Development Environment

Step 11:
Click on the Run button.

Setup Presto SQL Development Environment


Wait until SERVER STARTED message and visit http://localhost:8080.

Setup Presto SQL Development Environment

Setup Presto SQL Development Environment


Step 12:
Open a terminal in presto source folder and change to directory to the presto-cli/target and run presto-cli.

Setup Presto SQL Development Environment

Execute the following SQL and see if you get similar results.
presto> show catalogs;
 Catalog 
---------
 system  
 tpcds   
 tpch    
(3 rows)

Congratulations! You have successfully setup Presto development environment. If you face any problems in setting up Presto development environment, feel free to comment below.
Previous
Next Post »

Contact Form

Name

Email *

Message *