1. Get rid of all advertisements and get unlimited access to documents by upgrading to Premium Membership. Upgrade to Premium Now and also get a Premium Badge!

A new open source ETL tool

Discussion in 'Coffee Corner' started by tgdabler, Oct 15, 2009.

  1. tgdabler

    tgdabler Active Member

    Messages:
    2
    Likes Received:
    0
    Trophy Points:
    55
    Hello,

    We are trying to find an ETL tool open source. Basically, we need our software to perform ETL, data migration and data synchronization.

    The program should not be used on larger projects. A few open source tools exist on the market.

    Some ideas? Thanks.
     
  2. Sadik

    Sadik Community Moderator Forum Guru

    Messages:
    1,906
    Likes Received:
    252
    Trophy Points:
    1,455
    And I think someone will magically come here and suggest Talend. You guys should write a serious review and post it in the Marketplace forum instead of again and again raising such silly thread which get moved anyway.
     
  3. Loisal

    Loisal Active Member

    Messages:
    5
    Likes Received:
    0
    Trophy Points:
    55
    Moving spatial data into Post gres isn’t difficult to script. Suppose you have the shape file ponies.shp in WGS84, and you want to dump it into Post GIS as ponies. A plain old batch script like this will do it for you. That’s really it. You should probably vacuum the table after load, so there’s another line. You might want to set some particular permissions, which is another line, or get the data from SDE, which is another line (sde2shp is the easiest – just copy the SDE BIN folder to the running machine and put it in your path). But worst case scenario – 5 lines of code. On Windows I have a bat file that takes 4 arguments so I can cram a shape file into Post gres without having to do anything rash like thinking.
     
  4. Sadik

    Sadik Community Moderator Forum Guru

    Messages:
    1,906
    Likes Received:
    252
    Trophy Points:
    1,455
    What are you talking about? Do you have the faintest of ideas what data migration means? Please do not junk post for showing links. Consider this a first warning.
     
  5. Meenakshi

    Meenakshi Active Member

    Messages:
    2
    Likes Received:
    0
    Trophy Points:
    55
    I have worked in Pentaho ETL(Kettle) long back. And from my experience it has great capabilities. Give it a try .