WhatsOn Web Scraper is now open source. If you make any improvements or write your own web scraper, please send them to me so I can post them on this site and keep others using WhatsOn.
The WhatsOn Web Scraper is written using MFC and requires Microsoft Visual Studio to build. It has a full GUI user interface and was written originally for the TVGuide.com website. The web scraper pulls data from a website and writes it in a format that is read by the WhatsOn conduit. The conduit pulls the data down to the Palm where it is displayed by WhatsOn.prc.
The web scraper must write the data to a simple text file. The format is quite simple. The file is composed of a series of lines seperated by line feeds. There are two types of valid lines. The first type is a label line. It is of the form:
Label: 2 KTVU
Where the words in bold must be written as they are so that the WhatsOn conduit can parse the data correctly. This line tells WhatsOn how to associate a label with a channel number. The label is displayed on the left side of the program grid. This is important since all program listing data must refer to program listings by channel number.
The second type is a program listing line. It is of the form:
Program: date starttime - endtime channel x title Descr: description.
The time fields are expressed using military time. The date is expressed as mm-dd-yyyy. The description fields at the end are optional. If these fields are omitted, then there is no description data available for this listing. Here as an example of a listing line:
Program: 5-21-2000 14:00 - 15:00 channel 2 Baywatch Hawaii Descr: In the 10th-season finale, tragedy hits Baywatch when Mitch (David Hasselhoff) gets in over his head while battling ecoterrorists out to leave their mark on Hawaii. Pamela Bach. Sean: Jason Brooks. Kekoa: Stacy Kamano.
The WhatsOn Conduit does not have good error checking. So, errors in the file format will produce unpredictable results. The ordering of the listing data may also effect how it is displayed on the Palm.
The web scraper writes its data to a file called WhatsOn.txt. This file is then placed in the Palm user's directory prior to hotsyncing. A likely path for this file is: c:/palm/username/whatson/whatson.txt. Simply place the file there, hotsync, and the data will be synced to your palm where it can be displayed by whatson.prc.