Tools

Flexter Data Liberator

Description

Flexter is an enterprise ETL tool for XML and JSON. It automatically converts XML to text, any relational database, a cloud database, or Hadoop/Spark (ORC, Parquet, Avro). It works with any industry data standard (FpML, ACORD, FIXML, ISO 20022, HL7 etc.). Once the conversion process has completed, data analysts and other data consumers can use SQL or standard reporting tools such as Tableau, Qlik, Looker etc. to easily query FpML data.

Key Features

 

Flexter completely automates the whole process of converting FpML files from XML to a relational format or Hadoop/Spark. No custom development is needed. This can save up to 80% of the overall conversion costs. You don’t need to hire external consultants with XML/JSON expertise. As one customer put it recently “You did in one day what would have taken us a year”.

Flexter can process FpML data in real time through its streaming engine. As soon as an event happens in the real world it can be analysed instantly with the help of Flexter.

Flexter eliminates project risk. We have seen many FpML conversion projects fail.

Flexter is big data ready. We have built scalability into Flexter from the ground up. Flexter can scale up and out across multiple servers. It can handle any volume of data and meet any SLA.

Flexter significantly shortens the duration of FpML conversion projects. Developers can focus on data analytics tasks that add value to the business rather than having to wrangle with XML.

Flexter can meet any service level agreement. In tests it was an astonishing 800 times faster than competing solutions.

Flexter handles different versions of FpML gracefully. You can compare different versions of the standard and generate upgrade scripts automatically. No coding needed.

Technical Details

- We support both individual XML/JSON files and batches of XML/JSON files in archives and compressed formats (zip, gzip etc.).

- We can pull XML/JSON files from network drives, (S)FTP servers, HDFS, S3, XMLTYPE/CLOB/BLOB, BJSON in databases etc.

- Flexter supports very large XML files

- We support any relational databases, e.g. Oracle, MS SQL Server, DB2, PostgreSQL, MySQL, Redshift, Snowflake, BigQuery etc.

- Flexter can be called from the command line or through its RESTful API.

- We support popular Big Data formats (ORC, Parquet, Avro) and Hadoop SQL engines, e.g. Hive, Drill, Impala etc.

- We support schema evolution to easily upgrade between different versions of FpML.

 

The Flexter platform consists of three pluggable modules:

Step 1: The Schema Analyser is a dedicated module that loads, parses, processes and stores the XML/JSON schema information in Flexter's internal metadata DB. This step is only required to be performed once for each schema to be processed. You can either supply an XSD or a representative sample of XML/JSON files for this step.

Step 2: Now that we know the exact layout of the source XML/JSON it is possible to generate the relational equivalent. Flexter's module, Mapping Generator generates the output schema layout and the mapping to it. Various optimisations of the target schema can be applied during this step to make the schema more compact.

Step 3: The XML/JSON Processor module takes the information generated from the two previous steps, processes the XML/JSON, and writes the data to the relational target schema.

 

Flexter is written in Scala on Apache Spark. 

All past and future versions of FpML are supported. We support the full XSD specification. 

Supported Versions of FpML

FpML 5.0
FpML 5.1
FpML 5.2
FpML 5.3
FpML 5.4
FpML 5.5
FpML 5.6
FpML 5.7
FpML 5.8
FpML 5.9
FpML 5.10

Website

sonra.io/flexter-for-xml/