|
|
|
## Software Dependencies
|
|
|
|
* JDK 1.7, Python 2.7, APSW 3.8 (http://rogerbinns.github.io/apsw/).
|
|
|
|
* Passwordless ssh.
|
|
|
|
|
|
|
|
## Installing from tarball
|
|
|
|
* After installing the software dependencies you can unpack the tarball by executing :<br>
|
|
|
|
```bash
|
|
|
|
$ mkdir ~/exareme
|
|
|
|
$ tar xf exareme-*tar.gz -C ~/exareme
|
|
|
|
$ cd ~/exareme
|
|
|
|
```
|
|
|
|
* Installation directory structure (under ~/exareme/ ) :<br>
|
|
|
|
* /bin/ - binary scripts.<br>
|
|
|
|
* /etc/ - configuration and properties files.<br>
|
|
|
|
* /var/ - logs and pids files.<br>
|
|
|
|
* /lib/ - exareme, madis libraries and dependencies.<br>
|
|
|
|
|
|
|
|
* Current release can operate in local or distributed mode. You can configure the system accordingly by set up the appropiate files in ~/exareme/etc/exareme/.
|
|
|
|
* exareme-env.sh exports the required environment variables like java etc.
|
|
|
|
* Local mode requires only the master ip which you can provide by set up the master file.
|
|
|
|
* Distributed mode requires to set up the master and workers files. Workers file format requires one worker ip per line.
|
|
|
|
* Example:<br>
|
|
|
|
```bash
|
|
|
|
$ cat ~/exareme/etc/exareme/master
|
|
|
|
192.168.0.1
|
|
|
|
$ cat ~/exareme/etc/exareme/workers
|
|
|
|
192.168.0.2
|
|
|
|
192.168.0.3
|
|
|
|
```
|
|
|
|
|
|
|
|
## Running Exareme
|
|
|
|
* In ~/exareme/bin directory, you can find the exareme-admin.sh script which is responsible for the administration tasks for both local/distributed mode.<br>
|
|
|
|
```bash
|
|
|
|
$ cd ~/exareme
|
|
|
|
$ ./bin/exareme-admin.sh --install #You can skip it in local mode.
|
|
|
|
$ ./bin/exareme-admin.sh --start
|
|
|
|
$ ./bin/exareme-admin.sh --status
|
|
|
|
$ ./bin/exareme-admin.sh --stop
|
|
|
|
$ ./bin/exareme-admin.sh --kill
|
|
|
|
$ ./bin/exareme-admin.sh --uninstall #You can skip it in local mode.
|
|
|
|
```
|
|
|
|
|
|
|
|
## Executing Queries
|
|
|
|
* In order to execute exareme queries you can use either our jdbc driver or the console. By executing `./bin/exareme-admin.sh --console` a madis console will be prompt along with some exa-like operators available.
|
|
|
|
* exaquery - submit an EXADFL query.
|
|
|
|
* exaqueryscript - submit an EXADFL queryscript.
|
|
|
|
* exaresult - returns result table.
|
|
|
|
|
|
|
|
* Exa-like operators requires host, port, db property.
|
|
|
|
* you can set this global properties by executing.<br>
|
|
|
|
` mterm> var ‘db’ ‘/data/demo/tpch’;`
|
|
|
|
* or you can set as operator property <br>
|
|
|
|
` mterm> exaresult ‘db:/data/demo/tpch’ nation;`
|
|
|
|
* host and port are optional and can be skipped if the console is running on the same host as master node.
|
|
|
|
|
|
|
|
### Example usage of exa-like operators:
|
|
|
|
##### Sample import
|
|
|
|
```bash
|
|
|
|
$ ./bin/exareme-admin.sh --console
|
|
|
|
mterm> exaquery ‘db:/path/to/db/’ distributed create table lineitem as external
|
|
|
|
select
|
|
|
|
cast(c1 as int) as l_orderkey,
|
|
|
|
cast(c2 as int) as l_partkey,
|
|
|
|
cast(c3 as int) as l_suppkey,
|
|
|
|
cast(c4 as int) as l_linenumber,
|
|
|
|
cast(c5 as int) as l_quantity,
|
|
|
|
cast(c6 as float) as l_extendedprice,
|
|
|
|
cast(c7 as float) as l_discount,
|
|
|
|
cast(c8 as float) as l_tax,
|
|
|
|
cast(c9 as text) as l_returnflag,
|
|
|
|
cast(c10 as text) as l_linestatus,
|
|
|
|
cast(c11 as text) as l_shipdate,
|
|
|
|
cast(c12 as text) as l_commitdate,
|
|
|
|
cast(c13 as text) as l_receiptdate,
|
|
|
|
cast(c14 as text) as l_shipinstruct,
|
|
|
|
cast(c15 as text) as l_shipmode,
|
|
|
|
cast(c16 as text) as l_comment
|
|
|
|
from (file '/path/to/datasets/lineitem.tbl.gz' delimiter:| fast:1);
|
|
|
|
```
|
|
|
|
##### Sample query
|
|
|
|
```bash
|
|
|
|
$ cat /path/to/queries/q6.sql
|
|
|
|
distributed create temporary table lineitem_partial to 1 as
|
|
|
|
select sum(l_extendedprice * l_discount) as revenue_PARTIAL
|
|
|
|
from lineitem
|
|
|
|
where l_shipdate >= '1994-01-01'
|
|
|
|
and l_shipdate < '1995-01-01'
|
|
|
|
and l_discount > 0.07
|
|
|
|
and l_discount < 0.09
|
|
|
|
and l_quantity < 24;
|
|
|
|
|
|
|
|
distributed create table q6_result as
|
|
|
|
select sum(revenue_PARTIAL) as revenue
|
|
|
|
from lineitem_partial;
|
|
|
|
|
|
|
|
$ ./bin/exareme-admin.sh --console
|
|
|
|
mterm> exaqueryscript ‘db:/path/to/db’ file ‘/path/to/queries/q6.sql’;
|
|
|
|
```
|
|
|
|
##### Sample result
|
|
|
|
```bash
|
|
|
|
$ ./bin/exareme-admin.sh --console
|
|
|
|
mterm> select * from (exaresult ‘db:/path/to/db’ q6_result) limit 2;
|
|
|
|
```
|
|
|
|
|