Format date in Batch yyyyMMdd

The usual approach to build a timestamp for a log files in a batch is to use standard Windows utilities date and time. However the format of the output depends on locale and it is almost not possible to make the script which runs on any machine. A solution might be to create a small java […]

Removing time part of a date fails in Kettle

Might be an interesting info. I encountered an error running a Pentaho Data Integration transformation. After simplification I got this: Generate Row creates one row with the date field DATETIME=1981-01-01. Calculator calculates field DATE using operation "Remove time from a date". And this trivial transformation failed with a weird error: Unexpected error : java.lang.IllegalArgumentException: MINUTE […]

How to Obfuscate Password for Command Line Kettle

Pentaho Kettle does not provide possibility to obfuscate password in batch files (though the connection passwords are obfuscated in XML files). You can write: kitchen.bat /rep repos /job test /user admin /pass admin but this will not work: kitchen.bat /rep repos /job test /user admin /pass "Encrypted 2be98afc86aa7f2e4cb79ce71da9fa6d4" Here "Encrypted 2be98afc86aa7f2e4cb79ce71da9fa6d4" is encrypted version of "admin". You can get […]

Loading Fact Tables with PyGramETL

The previous post described how to load dimension tables with PyGramETL. This one is about fact tables. The source CVS file has the following structure: cust_id;res_date;future_guests;future_revenue 105;2008/04/01 00:00:00.000;2;300 106;2007/04/17 00:00:00.000;4;380 106;2007/04/17 00:00:00.000;4;768 … Fact Table CREATE TABLE reservation_fact( customer_sk INT, reservation_date DATETIME, future_guests FLOAT, future_revenue FLOAT ) Kettle PyGramETL

Loading Dimension Tables with PyGramETL

There is a promising Python framework for ETL – PyGramETL (http://pygrametl.org/) This post demonstrates an example how to import data from a CSV file and load it into a dimension table using PyGramETL in comparison with the same functionality implemented in a Kettle transformation. The input file customer.txt has the following structure: customer_id;first_name;last_name;age;phone_number;address 101;Paul;Brendt;19;(212) 555 […]