iBeg2 – Super Fast Super Efficient Diffing toolkit for Game Developers

Don’t be a Dick is key to our Manifesto and we live to make Games More Awesome. So…

Every studio has at least one of these challenges:

• They’re distributed
• QA/QC are in a different timezone
• Ensuring everybody is testing the same thing is hard
• Getting the correct game build to everybody is a logistical nightmare
• Moving big game builds across the internet can get super expensive super quick
• They love collaboration but not the album between Metallica and Lou Reed

We created (with our amazing board advisor, and tech director veteran, Fred Gill) the ultimate tool for studios – it diffs game builds super quick, it creates patch files that are super small and it allows you to get the right build into the right hands at the right time.

We’ve had several developers testing iBeg2 for months. Two of these developers have integrated iBeg2 into their build pipelines, and one of those developers estimates they will save over $150,000 in build distribution costs as well as getting builds to their team and QA significantly faster.

And the cost you ask? Free! We merely ask you to make a generous charitable donation (we love Antidote Gamers, GamesAid, Safe in our World, and Special Effect but we’re happy for any charity to gain if they’re not in your favourites list).

Why free? Because we’re privileged to be a community of developers all building our artform and imbuing it with magic. We find iBeg2 makes DevOps and TDs happy – and we love making people happy – that’s why we all pursued a “career” in games.

Plus iBeg2 works beautifully with our Studio Wizards toolkits and processes so we can make you even more efficient with your Build and Release cycle. We know it works because we dog fooded it to fix our clients problems.

iBeg2 is already integrated into a number of AAA game developer’s build pipelines. In the meantime our Code Wizards team has been busy building out a secure Cloud-based Build Distribution system for PC and consoles which we’d love to talk to you about too. We ran out of puns (not really – we never run out of puns) and so we simply call this “Build Manager”.

Key Features

• Linux or Windows capable
• Understands games builds and how these change in development
• Uber quick diffing
• Super clever algorithm which optimises how it creates a patch file depending on sizes, shape, type and difference of files
• Has a really pun-able name which any Brits will thorough enjoy – if you don’t then i Beg 2 Differ

Any Exclusions or Catches?

• Need to be a game studio
• Need to be good people
• Need to love games
• Need to optimise your build and release process
• Doesn’t include AI or Web3 or Big Data
• NEW: Doesn’t include the “most beautiful word in the world” -> ZERO Tariffs guaranteed!
• Please pass the word on – we want our developers to know their work is loved & used so please ask any new studios to send us an email and we’ll share it with them

How do i get it?

• Open your email application
• Send an email to ibeg2@codewizards.io
• Get ready for some amazing diff tooling (disclaimer: this is still manual for now so we’ll message you to verify you’re real and then sort out how we get it to you 🙂 )
• We won’t add you to a marketing campaign but we will email you with info on the iBeg2 details updates and what else it can work with (and maybe occasionally a good news story that we think is helpful to other devs)

And the License is..

(C) Code Wizards Limited 2025

iBeg2 License

[NOTE: This license is a modified form of the MIT license]

Permission is hereby granted, free of charge, to any person being granted a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, and merge copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

You swear that you will not be evil and will give money to a charity as a payback for this Software.

You acknowledge our rights to use terrible puns, “Dad Jokes”, and quips even if it makes you shudder.

You will not distribute, publish, sell or re-sell this Software either standalone or as part of another package outside of your own studio without obtaining express permission of Code Wizards Limited.

You will feedback on the Software and give us good news stories so we can all bask in goodness.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

The iBeg2 Manual

Why

Distributing the right builds to the right people has always been a key challenge to game developers.

iBeg2 was created to help game developers achieve the right build on the right platforms to the right people – quickly, efficiently and reliably.

Obviously, there are alternatives and many developers are using Steam to distribute builds to remote workers for testing and feedback as Valve’s CDN and build patching system is very good; typically, a Steam patch is 5%-15% of the total build size, and a patch can be downloaded and patched in 15 minutes with even a poor Internet connection. Valve’s patching system can also be used to revert back to the previous version quickly (usually a similar-sized patch) should the latest-and-greatest build not be suitable for testing, which maximises team test effectiveness on PC.

There are several issues with using Steam for distribution, including configuring correctly for security, and managing branches. Critically though, because PC builds are quicker to get into the hands of the team, it can mean teams focus on the PC build and miss critical testing on consoles.

iBeg2.exe is a core element of Build Wizard, a secure build distribution system that uses cloud infrastructure to get patches (differential builds) close to your team for all platforms you develop for.

Pre-Requisites

• 64-bit Windows or Linux OS
• An AVX2-capable CPU (checked on startup)
• Tea and Biscuits

Attribution and Kudos

iBeg2 is based on the amazing rsync algorithm by Andrew Tridgell. It also has some Code Wizards magic in there – additional compression that it applies as a result of having full access to both the “OLD” build data and the “NEW” build data. iBeg2 also uses the blazingly-fast xxHash by Yann Collet for generating 128-bit hashes, and uses the excellent 7zip by Igor Pavlov (when an excellent wheel already exists, we’ll use it rather than reinventing it) for an additional layer of lossless compression.

iBeg2 Command Line

There are three functions, and two modes of operation for iBeg2. The format of the command line is:

iBeg2 <command> <switches> <OLD file/directory> <NEW file/directory> <file>

Commands

c	Perform differential compression on a set of files / directories
d	Perform differential decompression on a set of files / directories
h	Output 128-bit hash for a file or directory hierarchy
b	Compare two binary files and display the offset of the first mismatch byte

If you run iBeg2 without parameters then the help is pretty good!

DIRECTORY Mode Compression & Decompression Example

This is the main use case, and indeed how Build Manager drives iBeg2.

Example generating a 7z of the differences between two directory hierarchies, olddir and newdir:

iBeg2 c olddir newdir oldnew.7z

iBeg2 generates a oldnew.7z diff containing all the changes between olddir and newdir

Example using olddir directory hierarchy and the 7z from above to newnewdir hierarchy:

iBeg2 d olddir newnewdir oldnew.7z

Note: if you binary compare newnewdir and newdir contents they will be an exact match.

iBeg2 Switches

-t<N>	Set how many threads should be used when creating a diff. Using -t2 or -t4 can give a significant speed-up, with diminishing returns above this.
-sx	Output extreme comp/decomp detail to stdout Note: this is legacy debug, and forces single-threaded compression
-cache	Cache some intermediate diff data in the OLDDIR directory, which makes future diffs against that OLDDIR directory quicker. Used by Build Manager to Example use cases: A – original build B – newer build C – even newer build If A->B diff is created with -cache, the process of diffing A->C becomes quicker when using -cache. Note: the cached data files sit side-by-side the original file in OLDDIR, with “_IB2#” appended to the file extension. iBeg2 doesn’t clear these files, so a simple “del *.*_IB2# /s” will delete them if necessary.
-i	Ignore any differential decomp errors (don’t stop on first error) This hasn’t been encountered (yet) with real build data
-k	Keep files rather than deleting i.e. when decompressing leave the master 7z around afterwards
-v	Verbose output to stdio [deprecated]
-o	Overwrite existing files / directories rather than stopping when an existing file is detected
-u	Ultra compression mode – takes longer, but resulting diffs are smaller (approximately ~1% smaller, but mileage does vary)
-bp<NN>	Set the bad diff percentage. If the DIFF file size is greater than this percentage of the NEW file size, don’t use the diff, but treat the file as new and just 7z to compress instead. The default is 95% as generally diff files are less compressible by 7z than the raw new file.
-@<CTRL>	Use a control (ctrl) file for finer control over comp / decomp. The control file is a text file with one entry per line, which can be a comment, or “IGNORE” or “ZIP” command – commands operate on files/directories, and are enclosed in double quotes, and support wildcards. Example for using a path – use Linux compatible style separator “/” – e.g. Engine/*.txt ) #The following ignores all .log files, and all files in Saved dir (and the dir too) IGNORE,”*.log” IGNORE,”Saved” #The following ZIPs rather than diffing the pdb (never diff well) ZIP,”*.pdb” #If debug is a dir, everything ignored (inc. debug directory) #If debug is a file, it is ignored IGNORE,”Debug”

Example generating newnewfile.bin from oldfile.bin and diff.gt64 (generated above):

Reporting Issues

We have tested iBeg2 extensively. If you should encounter any issues, or have requests for new features, please email: ibeg2@codewizards.io.

Non-Directory Modes

NON-DIRECTORY Modes

These features / modes were created to help with the development of iBeg2. They aren’t the expected use-case, but if you have a use for them, that’s GREAT!

• Single-file diff mode
  • Generate a DIFF file  from an OLDFILE and NEWFILE
  • Reconstruct a NEWFILE from an OLDFILE and DIFF file
• Hash display mode
  • Show 128-bit file hash for a single file
  • Show 128-bit file hashes and file sizes for a directory and subdirectories
• Binary compare mode
  • Compare and display first offset where two files differ

Note: the two files do not have to be the same size!

FILE Mode Compression & Decompression Example

Example of a diff.gt64 difference file being generated from oldfile.bin and newfile.bin:

iBeg2 c oldfile.bin newfile.bin diff.ib2

Note: There is no guarantee that the diff.ib2 will be smaller than newfile.bin. If that’s the case it’s a poor diff and you might as well just 7zip the newfile.bin!

Example generating newnewfile.bin from oldfile.bin and diff.gt64 (generated above): 

iBeg2 d oldfile.bin newnewfile.bin diff.ib2

Note: if you binary compare newnewfile.bin and newfile.bin they will be exactly the same.

Hash Mode Examples

Example generating 128-bit hash for a given file:

iBeg2 h file.bin

Example generating 128-bit hashes for all files in a directory (and subdirectories):

iBeg2 h dir

Binary Compare & Hash Mode Examples

Example comparing two binary files to find out the offset where they first differ:

iBeg2 h fileA.bin fileB.bin

HISTORIC DECISIONS MADE

• iBeg2 v2.xx is not backward compatible with V1.xx
• The largest OLD or NEW file that can be diffed is 128GB.
  • All files in the “multifile” file bucket must be less than 128GB.
• Need AVX2-capable CPU (for speed).
• 7z is called as an external program
• OLD and NEW must be different directories when decompressing – why?
  • OLD is not affected should there be a problem de-diffing.
  • Historically developers have lots of disk space.
  • Quick switching between NEW and OLD, should the NEW build be unstable.
• Comparisons on paths and filenames are done lower-case
  • Copies, Moves and Diffs follow the case of files in NEW in directory mode.
• iBeg2 decompression has lots of out-of-bounds checking to stop hijacking / exploits.
  • The input has been extensively fuzzed to validate the checks

FILE Mode Compression & Decompression Example

DIRECTORY Mode Algorithm – Diffing

For the geeks!

Most game builds don’t change that much between versions. iBeg2 uses that to maximise its effectiveness when generating a diff:

• Beg2 builds a list of all files in the OLD and NEW directories
• Pass #1 – check files in the same relative positions in OLD and NEW
  • For files with same filename
    • If the files are the same size, check their 128-bit hashes
      • If the same, mark as “copy”
    • Different sizes – mark as “diff”
• Pass #2 – “find moved files”
  • Run through all files in OLD putting into lists according to size
  • See if any file in NEW that has no action matches any OLD size
    • Check 128-bit hashes – if the same, mark as “move”
• Pass #3 – “find moved diffs”
  • Run through all files in NEW not marked with an action
    • If filename match in OLD (and unique in OLD), mark “diff move”
• Treat anything not marked with an action in NEW as actually “new”
  • Generate 128-bit hashes for all “new” files if they don’t already have one
• Now perform diffing for “diff” and “diff move” files

Finally:

• Generate the 128-bit hash for a “key” file in the OLD directory
• Get details of NEW directory – total files, total directories, and total size
• Generate a text file (masterIBeg2.txt) with instructions: copy, move, diff, diff-move
  1. Include key file, and details (#2 above) on the NEW directory
• 7z the diffs
• 7z all “new” files (those that are new in NEW, those too small, those that diffed badly)
• 7z the files from #3 – #5 into a single 7z file for transfer to clients for decompression

DIRECTORY Mode Algorithm – De-Diffing

• Unzip the master .7z file into the three files in the users temp directory
• Get the “key” file information from the masterIBeg2.txt file and check hash
  • Fail – stop as “olddir” very likely wrong reference point!
• Unzip the difffiles.7z into NEW
• Unzip the newfiles.7z into NEW
• Run through masterIBeg2.txt performing actions
  • “diff” and “diff move” – de-diff and check file size and 128-bit hash
  • “copy” – copy from OLD to NEW and check 128-bit hash
  • “move” – copy from OLD into new position in NEW, check 128-bit hash
  • “new” – check the 128-bit hash of the new file
    • This is “belt and braces” as 7z will have reported issues earlier!
• Once completed, total up all the files and directories, and file sizes, and ensure they match what was stored in masterIBeg2.txt.

NOTES

Why does v2.66 exist though?

• iBeg2 uses several retail games (Apex, Fortnite, ARC Raiders, etc.) as part of my test data. In 2025 Epic changed the way Fortnite was packaged, and instead of a couple of monolithic archive files (.ucas), they switched to many .ucas files of approximately 4GB in size – data moved extensively between archive files as new data added to the builds. iBeg2 didn’t handle this well as it had a heuristic where it tried to delta between single files (it one NEW file based on a single OLD file and a diff). iBeg2 v2.66 does multifile-diffing, where a single file in NEW is diffed against a bucket of files in OLDDIR (all files in the bucket have the same file extension). An exemple Fortnite patch went from 14.55GB in iBeg2 v1.54 down to 6.23GB in iBeg2 v2.66 (versus Epic’s patch size of 11.73GB).
• Caching was changed to facilitate side-by-side cache files (cache files reside in the OLD directory structure) rather than the cache files being in a system temporary directory – it makes managing old versions easier.
• It’s slightly faster generating the delta – a few core loop optimisations, and more elements are multi-threaded.