After reading through the post on diff’ing the native library’s , i thought i’d make a basic design for a incremental “cache” grabber,
here’s what i’d do:
Make a program that’s able to fetch files from a JS5 server:
Easy, already done by loads of people for there model grabbers, this would involve sending fake requests to a runescape or funorb server,
and parsing the responses into files.
Make a library for parsing JS5 containers and JS5 multiple file archive’s:
Can be done two ways: ripping client code and making your own, the containers are simplistic files with a header consisting of compression type, raw size and decompressed size, followed by the compressed data, the multiple file archives are a bit harder to decode as they have NO headers, the “header’s” for these files are in the reference table.
Make a program which parses the reference table’s and diff’s the CRC32 hashes in there for each file:
Diffing the reference table’s will involve two checks, grab reftable 255 (uKeys) and diff it with the last known version, if not changed:abort,else grab all the reftables and diff them, if there the same : abort, else parse them and check what files changed, and dump them to a list of some kind.
Make it copy the base “cache” (previous revision) and let it update the files that changed:
Easy, copy the old datastore from the last change and read the changelog , then grab those files and store them.
