Skip to content

Projects

A sample of interesting projects I've worked on, some of which I have expanded on to include sequence diagrams as a visual aid.

EPG CDR Processor

[Python | python.org] [MongoDB | mongodb.com] [Multiprocessing | docs.python.org] [Systemd | systemd.io] [ASN.1 | ncbi.nlm.nih.gov] [XML | w3.org] [YAML | yaml.org]

Developed a Python application to efficiently process and analyse Call Detail Records (CDRs). The tool parses XML-converted CDRs, extracting data points such as MSISDN, IMSI, network type, and traffic volumes, then stores these records in MongoDB for data retrieval and analysis.

sequenceDiagram
    participant EPG as Packet Gateway
    participant Mediation as Mediation Platform
    participant Files as CDR File Storage
    participant Decoder as CDR Decoder
    participant Processor as CDR Processor
    participant MongoDB as MongoDB
    EPG->>Mediation: SFTP Transfer to mediation
    Mediation->>Files: SFTP Transfer to file storage
    Files->>Decoder: ASN.1 to XML decoding
    Decoder->>Processor: Parsing decoded CDRs 
    Processor->>MongoDB: Database storage
  • Custom parsing of hexadecimal data for accurate location and timestamp information.
  • Multi-processing to handle large datasets concurrently (using python multiprocessing module).
  • Automated file management to prevent duplicate processing.
  • Error logging and handling for operational reliability.

OLM Processor

[Python | python.org] [MongoDB | mongodb.com] [Multiprocessing | docs.python.org] [Systemd | systemd.io] [YAML | yaml.org]

Designed and implemented a Python-based system for processing Online Mediation platform data, managing the collection and decoding of logs from various network streams.

sequenceDiagram
    participant OLM as Online Mediation Platform
    participant OLM_Collector as OLM Collector
    participant OLM_Decoder as OLM Decoder
    participant OLM_Processor as OLM Processor
    participant MongoDB as MongoDB
    OLM->>OLM_Collector: SFTP collection
    OLM_Collector->>OLM_Decoder: Decoding
    OLM_Decoder->>OLM_Processor: Processing
    OLM_Processor->>MongoDB: Storage
  • Collector utilises SFTP to fetch log files from multiple predefined streams, organizing and managing these files across local directories for processing.
  • Decoder parses log filenames and content to extract timestamps, stream identifiers and MSISDN.
  • Processor: Processes decoded files for storage in MongoDB.

Field Ops Portal

[Python | python.org] [Django | django.org] [Django OTP | github.com] [Django Two-Factor Authentication | pypi.org] [Celery | celery.org] [Nginx | nginx.org] [LDAP | python-ldap.org] [Rsyslog | rsyslog.com]

The main goal of this project was provide a facility that would allow providing customers with appointments in real time, without having to check back on availability, as this was provided in real time. This also included real time integration of CRM platform to retrieve and store relevant work order information.

The approach was to create actual empty time slots per engineer and have this allocated to appointments, this avoided any chance of double booking.

The system managed engineers, activities, time slots and appointments which were assigned to available time slots.

For appointment confirmation and reminders, Celery was used to schedule emails based on current date and appointment dates.

For internal use the Django Admin interface including significant modifications to show notes and time slot usage visually utilising Chart.js.

FTTH Migration Portal

TODO brief description

IMS Migration Portal

TODO brief description

MVNO SIM Activator

[Python | python.org] [XML SOAP | w3schools.com] [Systemd | systemd.io]

An automated system designed to manage and activate SIM cards by interfacing with a CRM (Customer Relationship Management) system. It monitors open activation events, processes relevant data, and executes necessary actions to complete activation workflows.

sequenceDiagram
    participant CRM as CRM Platform
    participant Events as Events
    participant WorkOrders as Work Orders
    participant Jobs as Jobs

    CRM->>Events: Event detection (SOAP API)
    Events->>WorkOrders: Extract access number data for activation
    WorkOrders->>Jobs: Activate work order job step and close event

MVNO Usage Database

[Python | python.org] [MySQL | mysql.com] [XML SOAP | w3schools.com] [Multiprocessing | docs.python.org] [YAML | yaml.org]

Developed a Python-based system for collecting, processing, and storing usage data of subscribers. Utilised MySQL for database management and interfaced with an SOAP API interface to fetch subscriber usage details.

sequenceDiagram
    participant CRS as CDR Reporting System
    participant Collector as Collector
    participant MySQL as MySQL Database

    CRS->>Collector: Collect specific data range for subscriber
    Collector->>MySQL: Store parsed data as usage records

Cockpitdecks

[Python | python.org] [Loupedeck | loupedeck.com] [Streamdeck | elgato.com] [X-Plane | x-plane.com] [Git | git-scm.com] [Github | github.com] [YAML | yaml.org]

Contributed to the development of Cockpitdecks, an open-source project providing a Python interface for controlling Loupedeck Live and other decks for integration with X-Plane flight simulation software.

  • Diagnosed and resolved compatibility issues with Loupedeck Live on newer hardware firmware models.
  • Contributed bug fixes and new features to Cockpitdecks repository.
  • Established a separate repository and documentation to host a comprehensive set of aircraft configurations for use with CockpitDecks.

SMS Gateway

[Python | python.org] [YAML | yaml.org] [Django | django.org] [Nginx | nginx.org]

Developed a solution to provide an API that could interface an SMS Centre. This comprised of a Portal (Django + PostgreSQL), a front end API (FastAPI) and background services (Python) that could interface directly with the SMS Centre using SMPP.

sequenceDiagram
    participant Enduser as End Users
    Enduser->>FrontendAPI: RESTful API Client
    participant FrontendAPI as External REST API
    participant Django as Billing & Messaging Engine
    participant Services as Services
    participant SMSC as SMS Centre
    FrontendAPI->>Django: Send and receive messages
    Services->>Django: Incoming messages and delivery reports
    Services->>SMSC: Transmit messages
    Django->>Services: Outgoing messages
    SMSC->>Services: Incoming messages and delivery reports

Django Rest Framework, Admin Interface, Signals used to adjust increase client balances on top up decrease on creating message.

Django Models made use of foreign keys, extensive use of attributes such as on_delete, help_text, validators, db_index.

Models using of clean and save functions to check for credit and save allowed adding dynamic data such as number of SMS parts required for a message.

Heavily customised admin views.

Developed an SMSC telnet library required to interface with SMS centre to retrieve information such as status of the SMS centre, throughput and licensing limits.

Delivery reports were recieved via SMPP connection and sent to the Portal via Django's REST framework API.

Services to handle incoming and outgoing messages were handled via separate listener and transmitter services connected via SMPP.

ETK (Provisioning Platform)

TODO expanded description

sequenceDiagram
    participant A as Flask Front End & CLI Interface
    participant B as Packages (interface libraries)
    participant C as DB Storage (MongoDB)
    participant D as External Interfaces
    A->>B: Load custom Python modules
    B->>C: interface configuration
    B->>D: Telnet/SOAP/RESTFul

RQ Active/Standby provisioning Provisioning of and presentation of Panic button data using SMS.

Check_MK Monitoring System

TODO

MVNO Number Porting

[Python | python.org] [telnetlib | docs.python.org] [MongoDB | mongodb.com] [Systemd | systemd.io] [YAML | yaml.org] [MML (man–machine language) | en.wikipedia.org] [Multiprocessing | docs.python.org]

Developed a Python-based MVNO number porting mediation service for a spanish MVNO to automate the provisioning of ported numbers. This that utilises SFTP for file collection, MongoDB for data storage and parsing of CSV/gzip data files. It also required interfacing with a Flexible Number Registry via Telnet and MML to retrieve existing subscriber translations and activate new ones. Multiprocessing capability was introduced allowing multiple activators to run in parallel.

sequenceDiagram
    participant AOPM as AOPM
    participant Collector as Collector
    participant FileStorage as File Storage
    participant Importer as Importer
    participant Database as Database (MongoDB)
    participant Processor as Processor
    participant Activator as Activator
    participant FNR as Flexible Number Registry

    AOPM->>Collector: SFTP transfer of files
    Collector->>FileStorage: Storage of collected files
    FileStorage->>Importer: Reading data from file storage
    Importer->>Database: Saving file data to database
    FileStorage->>Processor: Translation of file data to subscriber translations
    Processor->>Database: Storing subscriber translations for activation
    Activator->>Database: Retrieving subscriber translations
    Activator->>FNR: Activation of subscriber translations

Modular Design: Employed a modular architecture (Collector, Importer, Processor, Activator); this allowed for easier maintenance, better scalability and improved reliability.