Presto SQL a massively parallel processing big-data engine grasps the attention of many big-data developers. This article is for those who like to set up a development environment for the Presto SQL community edition. The below-mentioned steps are applicable for any Presto variations including Presto DB with class names and file names replaced by equivalent names.
Requirements:
- Java Development Kit 8 (after March 2020, Presto SQL is using Java 11)
- IntelliJ IDEA Community/Ultimate Edition
- Latest Apache Maven (Older versions may not build Presto)
- Git
Step 01:
Clone the master branch from the official Git repository
git clone https://github.com/prestosql/presto.gitStep 02:
Change the directory to the presto folder and build the project ignoring tests (Presto tests require docker and may take hours to complete)
cd presto
mvn clean install -DskipTestsWait until the build to complete successfully.
Running Presto from IntelliJ requires the local maven repository to be available at
~/.m2/repository. If your local repository is somewhere else, create a soft link folder at ~/.m2/repository pointing to the original locationStep 03:
Open IntelliJ IDEA and import the presto folder as a project.
Once the project is imported, it should look like this:
Step 05 [Optional]:
If you are going to test memory-intensive queries while running Presto from IntelliJ, it is advised to increase the JVM maximum heap size. Processes started from IntelliJ IDEA are limited by the max heap size of IntelliJ IDEA. Therefore, increase the heap size of IntelliJ IDEA from Help → Edit Custom VM Options...
-Xms2G
-Xmx15G
-XX:ReservedCodeCacheSize=240m
-XX:+UseConcMarkSweepGC
-XX:SoftRefLRUPolicyMSPerMB=50
-ea
-XX:CICompilerCount=2
-Dsun.io.useCanonPrefixCache=false
-Djava.net.preferIPv4Stack=true
-Djdk.http.auth.tunneling.disabledSchemes=""
-XX:+HeapDumpOnOutOfMemoryError
-XX:-OmitStackTraceInFastThrow
-Djdk.attach.allowAttachSelf
-Dawt.useSystemAAFontSettings=lcd
-Dsun.java2d.renderer=sun.java2d.marlin.MarlinRenderingEngine
-Dsun.tools.attach.tmp.only=trueModify the Xms and Xmx properties according to the available memory and restart IntelliJ IDEA to apply the change.
Step 06:
Open the Project Structure from the File menu (Shotcut is Ctrl + Alt + Shift + S) and make sure that the Java 8 is selected as the Project SDK as shown below. For the latest Presto source (after March 2020), choose Java 11.
If you are using Java 11 to build, Presto, disable the Use --rebase option for corss-compilation (Java 9 and later) from Settings to avoid an issue in resolving sun.misc package.
Step 07:
From the Build menu, select Build Project (Ctrl + F9) to build the project from IntelliJ IDEA.
Delete all unnecessary catalog properties from the presto-main/etc/catalog folder. For a hello world execution, tpch.properties and tpcds.properties are good enough. Catalog property with invalid configuration will prevent Presto server from starting up.
Step 09:
Click on Add Configuration... option
Step 10:
Select the Application template from the left side panel and enter the following details:
Main class:
io.prestosql.server.PrestoServerVM options:
-ea -XX:+UseG1GC -XX:G1HeapRegionSize=32M -XX:+UseGCOverheadLimit -XX:+ExplicitGCInvokesConcurrent -Xmx2G -Dconfig=etc/config.properties -Dlog.levels-file=etc/log.properties -Djdk.attach.allowAttachSelf=trueIncrease the max heap memory in this argument if you are planning to execute memory intensive queries. The max heap size of Presto should be less than the max heap size of IntelliJ IDEA set in Setp 05.
Working directory:
$MODULE_DIR$Use classpath of module:
presto-main
Step 11:
Click on the Run button.
Wait until SERVER STARTED message and visit http://localhost:8080.
Step 12:
Open a terminal in presto source folder and change to directory to the presto-cli/target and run presto-cli.
Execute the following SQL and see if you get similar results.
presto> show catalogs;
Catalog
---------
system
tpcds
tpch
(3 rows)Congratulations! You have successfully setup Presto development environment. If you face any problems in setting up Presto development environment, feel free to comment below.
EmoticonEmoticon