Filter Stage:
Filter stage is a processing stage used to filter database based on filter condition.
The filter stage is configured by creating expression in the where clause.
Scenario1:Check for empty values in the customer name field.We are reading from sequential file and hence we should check for empty value instead of null.
Scenario 2:Comparing incoming fields.check transaction date falls between strt_dt and end_dt and filter those records.
Input Data:
| CUSTOMER_ID | CUSTOMER_NAME | TRANSACTION_DT | STR_DT | END_DT |
| 1 | UMA | 1/1/2010 | 5/20/2010 | 12/20/2010 |
| 1 | UMA | 5/28/2011 | 5/20/2010 | 12/20/2010 |
Output:
| CUSTOMER_ID | CUSTOMER_NAME | TRANSACTION_DT | STR_DT | END_DT |
| 1 | UMA | 5/28/2011 | 5/20/2010 | 12/20/2010 |
Reject:
| CUSTOMER_ID | CUSTOMER_NAME | TRANSACTION_DT | STR_DT | END_DT |
| 1 | UMA | 1/1/2010 | 5/20/2010 | 12/20/2010 |
Partition data based on CUSTOMER_ID to make sure all rows with same key values process on the same node.
Condition : where TRANSACTION_DT Between STRT_DT and END_DT
Actual Output:
Actual Reject Data:
Scenario 3:Evaluating input column data
ex:Where CUSTOMER_NAME=’UMA’ AND CUSTOMER_ID=’1′
Output :
Reject :
we can also use the relational operators aswell and you can also check for more articles Visit:www.datastage.in
ReplyDeletehow would you hardcode a date in a filter stage where clause (for testing purposes).
ReplyDelete