Configuring Remote Compiling

Some of us have multiple computers laying around that are doing nothing most of the time. Maybe it’s a file server that’s got an i5 in it, maybe it’s slower than your main computer. Heck it could even be a remote server sitting in a data center somewhere. The main point is you’re tired of using your computer to compile your Source Engine levels. It’d be nice to be able to offload the compile to those other machines so your workstation is free for use. Let’s get started!

There are a few steps that you’ll need to do so this all works.

  1. Create an environment for multiple users to access a level’s content
  2. Configure Remote Server
  3. Create Scripts
  4. Invoking a Compile To the Remote Machine
    1. PSExec (INSECURE OVER WAN/External)
    2. RDP (Works for LAN/WAN but is slower than scripts)
    3. Telnet (INSECURE OVER WAN/External)
    4. SSH (Ideal for WAN)

Here are a few things to note

  1. You’re going to have to know a small amount of networking.
  2. You have to create a bat file for each level you’re working on.
    1. Most of us only work on one level at a time anyway. This is no big deal.
    2. You CAN set up variables, but I’m not going to show you that since my current method works perfect for me.
  3. You’re going to need the remote credentials for the server you’re accessing.
    1. Remote credentials are required to send commands to ANY remote machine.
  4. If you’re using subst drives (I currently do) you cannot use those, as it won’t see them as drives and will result in a failed execute.

Create an environment for multiple users to access a level’s content

We need a platform where we can keep all of our custom content in sync between our workstation and our compile machine. We are going to set up the Custom Content Without Contamination environment that I’ve written about. Using a cloud sync service (DropBox, Google Drive, OneDrive) will work for both LAN and WAN environments. For this to work you have to have ALL of your project’ss content in the cloud service. Including the VMF files. There is no exception.

Configure Remote Server

First we need to get all the required files on our remote machine. Essentially the remote machine needs to be setup as if it were your workstation.

  1. Install Steam on the remote machine.
  2. Install the game you want to compile for, along with the SDK (if any).
  3. Install your cloud sync service on the remote computer and sync all your project files.
  4. Custom Content Without Contamination on remote server. This includes creating the GAME_dev folder with a custom gameinfo.txt mounting all the required content. 

Create Remote Scripts

I’m a simple man, so I use simple scripts. We are going to create 2 .bat files for the remote server.

  1. Remote Machine – Start Compile
  2. Remote Machine – Cancel Compile

Store ALL of these bat files in your cloud account. I suggest in the following path: Level Design\CSGO\PROJECT\Remote

My path is: D:\Dropbox\Level Design\CSGO\Collaborations\Titan\Remote

Remote Machine – Start Compile

This script is straight forward. It’s just going to fire the commands that our F9 (Run Map) menu from Hammer would normally fire.

A standard compile when Custom Content Without Contamination is configured will consist of 6 – we are only using 5, since we don’t want to copy the file on the remote server.

The required commands, with their switches in order are (please note you can add -threads # and other commands if you’d like):

  1. vbsp.exe -game “GAMEINFO.TXT_LOCATION” “PATH TO VMF“
  2. vvis.exe -game “GAMEINFO.TXT_LOCATION” “PATH TO VMF“
  3. vrad.exe -hdr -textureshadows -StaticPropLighting -StaticPropPolys -final -game “GAMEINFO.TXT_LOCATION” “PATH TO VMF“
  4. BSPZIPWalker.exe “CONTENT FOLDER TO WALK” “PATH TO MAP-FILELIST.TXT” (BSPZIPwalker.exe is a custom application written by BenVlodgi to preform the same functions as DityHelper.jar. It functions exactly the same as DirtyHelper.jar input and output wise. We no longer need Java!) BSPZIPWalker.exe can be downloaded here.
  5. BZPZIP.exe -addorupdatelist “INPUT BSP” “PATH TO MAP-FILELIST.TXT” “OUTPUT BSP” -game “GAMEINFO.TXT_LOCATION“

Here is my Remote Compile.bat file that is run on my compile server. You can pretty much copy this, update/remote the -threads command to your liking. Then update paths and names to your paths.

[php]”E:\Steam\steamapps\common\Counter-Strike Global Offensive\bin\vbsp.exe” -game “E:\Steam\steamapps\common\Counter-Strike Global Offensive\csgo_dev” “E:\Dropbox\Level Design\CSGO\Collaborations\Titan\raw content\maps\de_titan.vmf” “E:\Steam\steamapps\common\Counter-Strike Global Offensive\bin\vvis.exe” -threads 7 -game “E:\Steam\steamapps\common\Counter-Strike Global Offensive\csgo_dev” “E:\Dropbox\Level Design\CSGO\Collaborations\Titan\raw content\maps\de_titan.vmf” “E:\Steam\steamapps\common\Counter-Strike Global Offensive\bin\vrad.exe” -threads 7 -hdr -textureshadows -StaticPropLighting -StaticPropPolys -final -game “E:\Steam\steamapps\common\Counter-Strike Global Offensive\csgo_dev” “E:\Dropbox\Level Design\CSGO\Collaborations\Titan\raw content\maps\de_titan.vmf” “E:\Dropbox\Level Design\tools\BSPZIP Walker\BSPZIPWalker.exe” “E:\Dropbox\Level Design\CSGO\Collaborations\titan\Content” “E:\Dropbox\Level Design\CSGO\Collaborations\titan\Raw Content\Maps\de_titan-filelist.txt” “E:\Steam\steamapps\common\Counter-Strike Global Offensive\bin\bspzip.exe” -addorupdatelist “E:\Dropbox\Level Design\CSGO\Collaborations\Titan\raw content\maps\de_titan.bsp” “E:\Dropbox\Level Design\CSGO\Collaborations\Titan\raw content\maps\de_titan-filelist.txt” “E:\Dropbox\Level Design\CSGO\Collaborations\Titan\raw content\maps\de_titan.bsp” -game “E:\Steam\steamapps\common\Counter-Strike Global Offensive\csgo” [/php]

Remote Machine – Cancel Compile

Soon after I setup the remote compile, I realized that I needed a way to cancel the compile. This bat file is stupid simple – it just force closes all the compiler programs 4 times each. If you only did one round of taskkill it would sometimes miss the exe as when you cancel VVIS it takes a second to VRAD to start. It does not hurt to just fire each 16 times.

Here is my Remote Cancel.bat file. /f switch specifies to forcefully terminate the process. /im switch specifies the image name of the process to be terminated.

[php] taskkill /f /im vbsp.exe taskkill /f /im vvis.exe taskkill /f /im vrad.exe taskkill /f /im bspzip.exe Repeat the above 4 commands 15 more times… [/php]

Invoking a Compile To the Remote Machine

You have a few choices when executing commands remotely. My favorite is PSExec when I have local access to the computer. In this guide I’m not going to show you how to configure SSH to machines. There are many free SSH servers out there that you can install and use. For this guide I’ll show you how to use PSExec, TelNet (Telnet Server removed from Windows 10, thanks MS), and RDP.

PSExec

This is a sysadmin tool that allows you to fire remote commands to other machines on the network. It’s not suggested to use this tool over a WAN connection, but it’s great for internal connections. First you’ll need to download the Sysinternals Suite that has PSExec included in it.

Extract PSExec.exe somewhere that it can live forever on YOUR workstation. Remember we use PSExec on our machine to tell the remote server what to do.

PSExec Remote Compile Invoke Bat File

This is the bat file on our workstation that will tell PSExec what to do so the remote machine can start the compile. Here are the contents of the bat file in order:

  1. PSExec.exe \\SERVER_IP -u USERNAME -p PASSWORD “PATH TO REMOTE COMPILE.BAT“
  2. pause
  3. xcopy /y “PATH ON WORKSTATION TO BSP.bsp” “PATH TO GAME MAPS FOLDER ON WORKSTATION.bsp*”

Here is my Remote Compile Invoke.bat file. The * at the end of the copy command is not a mistake. It tells xcopy to always assume we are copying a file. 

[php] “D:\Dropbox\Programs\Tech Programs\Misc\Sysinternals Suite\PsExec.exe” \\172.16.21.1 -u administrator -p AYYYLMAO “E:\Dropbox\Level Design\CSGO\Collaborations\Titan\Remote\Remote Compile.bat” pause xcopy /y “D:\Dropbox\Level Design\CSGO\Collaborations\Titan\raw content\maps\de_titan.bsp” “C:\Program Files (x86)\Steam\steamapps\common\Counter-Strike Global Offensive\csgo\maps\de_titan.bsp*” [/php]

When running this script the compile executes. Upon completion the script pauses itself and waits for a key press. During this time the cloud service should transfer the compiled BSP back to our machine. Once I notice that DropBox has downloaded the compiled BSP, I press any key and the BSP is moved into my Maps folder. If you want to manually move the map after a compile, you can leave out the pause and xcopy command.

PSExec Remote Cancel Invoke Bat File

Copy the first PSExec line from the previous bat file, then change the path to be Remote Cancel.bat instead of Remote Compile.bat.

PSExec Remote Bat File Administrator Access

Lastly for some reason PSExec needs to be run as administrator on your local machine to work. After your invoke bat files are created, create a shortcut and make them always run as administrator.

  1. Select the Invoke Bat file.
  2. Right-Click and drag then release.
  3. Select Create shortcuts here.2016-07-27_02-41-00
  4. Right click on the new shortcut and select Properties.
  5. Click Advanced…dllhost_2016-07-27_02-43-07
  6. Check the Run as administrator box then click OK and OK again.
    dllhost_2016-07-27_02-43-44

That’s it for PSExec. Just double-click on one of the shortcuts to start or stop your remote compile!

COMPILE_OUTPUT

NOTE: If you want to start a remote compile of a save you just preformed on your workstation. Save the file in Hammer and wait for it to be uploaded to your cloud storage. It’s best to wait a moment or two so it downloads on your remote server, then start the remote compile shortcut.

TelNet / SSH

TelNet and SSH are really easy to use. It just gives you remote CMD access to a computer where you can run command line commands.

To use SSH you’ll have to install and configure a SSH server from the internet. OpenSSH is one I’ve used in the past.

To enable a TelNet server for your remote server, just Google Install TelNet server YOUR OS. (This link will probably be what you find. https://technet.microsoft.com/en-us/library/cc770501%28v=ws.10%29.aspx?f=255&MSPPError=-2147217396)

If your compile box is running Windows 10, the TelNet Server feature was removed. If you were going to use TelNet, you could probably just use PSExec instead.

After TelNet or SSH are installed they pretty much both function the same, with the main difference that SSH is secure and TelNet is not. This is why SSH should be used over TelNet if you’re connecting to an external server.

After your TelNet or SSH server is setup and you connect using a client like PuTTY, just run your Remote Compile.bat file.
putty_2016-07-27_03-00-37

RDP – Remote Desktop Protocol

RDP is a feature built into the Pro versions of Windows that allows remote console access to a computer. Once enabled you just login to the computer with the remote credentials and it’s like you’re sitting in front of it.

Enabling RDP has been the same for a while on all Windows Operating systems. You can Google how to enable it, but this guide should get you going – https://support.microsoft.com/en-us/help/17463/windows-7-connect-to-another-computer-remote-desktop-connection

After RDP is enabled and you’re connected, just run the Remote Compile.bat and the compile will start.

mstsc_2016-07-27_03-06-07

That’s it. You now have a remote compile environment for Hammer!