file url for the partitioned sandbox

Support questions related to CloverETL Server

sameerj365
Posts: 17
Joined: Tue Mar 25, 2014 8:39 am

file url for the partitioned sandbox

Postby sameerj365 » Thu Jun 19, 2014 10:51 am

Hi,

Please help me with the URL for the partitioned sandbox.

Flow :

Input DB table --> Cluster Partition --> Universal Data Writer

Here my o/p file will refer to 2-node partitioned sandbox. What will be the url for the data file so that it will write partition data into two seperate directories.

imriskal
Posts: 383
Joined: Wed Aug 15, 2012 8:18 am

Re: file url for the partitioned sandbox

Postby imriskal » Mon Jun 23, 2014 3:13 pm

Hi sameerj,

URLs are given by your settings in partitioned sandbox, see my screenshot:
partitioned.png
partitioned.png (30.44 KiB) Viewed 1670 times

I also recommend you to read this article in our documentation: http://doc.cloveretl.com/documentation/ ... ssing.html The whole issue is described quite nicely there.

You can use also sandboxes.home.partitioned property if you want to unify them, see: http://doc.cloveretl.com/documentation/ ... rties.html

Regards,
---
Lubos Imriska
CloverCARE Support
CloverETL | Rapid Data Integration

Visit us online at http://www.cloveretl.com
How to speed up communication with CloverCARE support

sameerj365
Posts: 17
Joined: Tue Mar 25, 2014 8:39 am

Re: file url for the partitioned sandbox

Postby sameerj365 » Tue Aug 19, 2014 11:25 am

Let me rephrase my question again.

I have a requirement to unload the data from the DB table and create a multifile. If I use DB -- ClusterPartition --- UniversalDataWriter (2 nodes allocation), then what would be the URL for the output file.

I have created partitioned SB and have defined it in the properties but my requirement is to write two seperate files for both the nodes in seperate directory under partitioned sb.

Let's say there are 1000 records then 500 records should be written to node1 directory and 500 records to node2 directory.

I have created shared file system so that both the cluster nodes can access it and within that shared file location, I have created two seperate directories for node 1 and node2.

Thank You

imriskal
Posts: 383
Joined: Wed Aug 15, 2012 8:18 am

Re: file url for the partitioned sandbox

Postby imriskal » Wed Aug 20, 2014 1:22 pm

Oh, I think I understand now. You can use this syntax:

sandbox://abcd/my_partitioned_file.csv

Using the example screenshot above, a half of the records would be saved in X:/sandboxes/shared/01/abcd/my_partitioned_file.csv on node01 and the rest would be in /opt/CloverETL/sandboxes/shared/02/abcd/my_partitioned_file.csv on node02.

Is this the result you expect?

By the way, I think the shared file system is not necessary in this situation. Both nodes can work on their own piece of data, they do not have to share the data. If you want to use a shared workspace, you can use shared sandbox instead of partitioned one.
---
Lubos Imriska
CloverCARE Support
CloverETL | Rapid Data Integration

Visit us online at http://www.cloveretl.com
How to speed up communication with CloverCARE support

sameerj365
Posts: 17
Joined: Tue Mar 25, 2014 8:39 am

Re: file url for the partitioned sandbox

Postby sameerj365 » Mon Sep 01, 2014 7:59 am

thank you for your help. I am able to write into seperate files.