FTPBeast Introduction   

 
     FTPBeast is a program I wrote in July of 2002, to demonstrate the ability to address complex business needs of data source integration and automation, applying a wide variety of techniques and protocols to create 'hands off' solutions.  Use the 'Select' menu above to see more.

      I hope you will use the 'Select' button above to read more about this project, see some presentation drawings and screen shots, etc.

      I chose to use the website logs of Yahoo! web hosting customers because my domains and the logs from them offer a good data source for demonstration.   The programming I applied here can of course be applied to any data source ( Internet / WAN / LAN ) from legacy apps, FTP sites, E-Commerce partners, other databases, etc.

      The logs are created by Yahoo! on their Unix servers, in a packed Unix '*.GZ' format.  FTPBeast gets them, archives them, unpacks them, extracts the data into a SQL database, integrates DNS canonical name lookups to add value to the data, stores that into SQL as well , then generates GUI reports, Web reports ( back out to my site on  Yahoo!'s web servers ), and status / confirmation emails.  All of this happens in less time than it takes to read about it, and with total 'hands off' server-side automation based on schedule.

 

  • Written 100 % in Visual Basic 6 - no 'third party' OCX's
  • Integration - Multi-protocol integration ( SQL, FTP, WinZip, SMTP, DNS lookup, etc )
  •       Combines various techniques and protocols for seamless integration of data sources
  •       Combines FTP with WinZip for 'unzipping' *.ZIP, *.GZ, etc files
  •       Applicable to E-Commerce, legacy app integration, etc
  •       Other protocols such as Telnet, etc can be added and totally automated, as needed
  • Automation - Complete 'hands off' schedule-triggered automation of : 
  •       Internet / WAN / LAN file synchronization, archiving, and retrieval 
  •       Complex data conversion tasks
  •        SQL 'Data Warehouse' based
  •        Invisible automation of external tools such as WinZip
  • Reports 
  •       GUI reporting on-screen, user controlled and filtered
  •       Web ( HTML ) reports exported to any website ( Internet / WAN / LAN )
  • Exports / Data Warehouse
  •        Reformats and exports the data originally obtained from an FTP session as CSV, ASCII, etc.
  •        SQL foundation makes the output part of the 'Total Enterprise Data Resource'