Data Integrations & ETL processes
I have been involved with many ETL processes to integrate connectivity between various applications and systems. This has involved
- data views
- comma delimited, tab delimited and fixed width files
- complex XML files (including delta mode)
- writing directly into the system database
- creating web-services
![](https://www.brentknigge.com//images/integrations/bpay.png)
Internet Banking Payment System
Wrote the CRN algorythm (with check digit) and integrated into the invoicing system so students had the option to pay their invoices online via BPay.
Developed a cronjob to automate the process of reading and loading the BPay transactions back into the finance system.
![](https://www.brentknigge.com//images/integrations/cyon.png)
Timetabling and examination scheduling software
Data views were provided for the timetabling, a web service connection via Moodle for examinations and my very own DE Application wrote data directly to the exam scheduler database for deferred exams.
![](https://www.brentknigge.com//images/integrations/alma.png)
Library System
I have built several advanced interfaces and add-ons to Alma.
- Address database (application and database design)
- Finance integration between Alma and a University finance system
- Complex scripts that are used to fix migration issues
![](https://www.brentknigge.com//images/integrations/rosetta.png)
Digital Presservation System
I have built sophisticated automation tools for Rosetta. These include:
- Reading and parsing their automated emails
- Using Codeception to perform tasks where no API was available
- Provide a scheduling and downloading tool
![](https://www.brentknigge.com//images/integrations/unicard.png)
Student ID Cards
I built a single view from the Student Information System that provided all the necessary information for the production of ID cards.
I also implemented a barcode algorythm for the ID cards that was integrated into the library system which also alleviated the manual process of activating the ID cards in the library.
![](https://www.brentknigge.com//images/integrations/rmlogo.png)
Research Management Solutions
I wrote a sophisticated routine to extract research data and upload the relevant information into the University's Academic Workload Planning System for each staff member.
This integration reduced a large amount of work that a staff member would have otherwise needed to perform manually.
![](https://www.brentknigge.com//images/integrations/moodle-logo.png)
Learning Management System
I built an easy to use secure web-service that allowed the Moodle integrators to access the required data from Oracle and SQL-Server databases using standard SQL statements.
Checks and balances were built in as part of the security where only select
could
be executed, and only on specific tables/views
![](https://www.brentknigge.com//images/integrations/adlib.png)
Collection Management Software
I have built an easy to use UI for the staff at the State Library that helps them immensly with their work. It allows for
- easy way to do bulk update to records
- automated formatting and grouping of columns for xml to xlsx conversions
- automated adlib and marc21 XML from ref codes
![](https://www.brentknigge.com//images/integrations/sars-logo.png)
Appointment Booking Software
I built a couple of views that the SARS-Grid application could connect to and extract the necessary data for their appointment booking software.
![](https://www.brentknigge.com//images/integrations/d2l-logo.png)
ePortfolio System
Relevant data from Banner was collated into a series of XML data files and sent to the ePortfolio server
![](https://www.brentknigge.com//images/integrations/allocate.png)
Timetabling and self allocation software
I built a series of views and flat file exports with an additional table that the vendor could update to control the semester codes that the views related to.
![](https://www.brentknigge.com//images/integrations/inplace.png)
Student Placement System
There were approximately 15 XML files that I built as part of the extract from the University Student System into Inplace.