LVL Lua Code Helper

[cbadal]

Looking at the swbf-unmunge thread by SleepKiller, Marth8880 requested a Lua decompiler.
And of course, I want one too.

I tried looking for a Lua decompiler last month and found 2 that didn't work on lua code compiled without debugging info (like is the case with the SWBF2 lua code).
I had no idea why one didn't exist. Lua seems like such a simple language.
I started looking into a paper called "A no Frills introduction to Lua 51 VM Instruction set" today to try to see if it would bee all that difficult to write one.

I got about 2 pages in and the author mentions that luac has a 'listing mode'. I tried out this listing mode and was quite surprised at just how easy the output is to understand.
Here is an example:

Hidden/Spoiler:

[code]-- cor1c_con
-- luac -l listing

main <(none):0> (20 instructions, 80 bytes at 02894830)
0 params, 2 stacks, 0 upvalues, 0 locals, 12 constants, 2 functions
1 [-] GETGLOBAL 0 1 ; ScriptCB_DoFile
2 [-] LOADK 1 2 ; "ObjectiveConquest"
3 [-] CALL 0 2 2
4 [-] SETGLOBAL 0 0 ; Conquest
5 [-] GETGLOBAL 0 1 ; ScriptCB_DoFile
6 [-] LOADK 1 3 ; "setup_teams"
7 [-] CALL 0 2 1
8 [-] LOADK 0 5 ; 1
9 [-] SETGLOBAL 0 4 ; ATT
10 [-] LOADK 0 7 ; 2
11 [-] SETGLOBAL 0 6 ; DEF
12 [-] GETGLOBAL 0 4 ; ATT
13 [-] SETGLOBAL 0 8 ; CIS
14 [-] GETGLOBAL 0 6 ; DEF
15 [-] SETGLOBAL 0 9 ; REP
16 [-] CLOSURE 0 0 ; 02894AA0
17 [-] SETGLOBAL 0 10 ; ScriptPostLoad
18 [-] CLOSURE 0 1 ; 02895AC0
19 [-] SETGLOBAL 0 11 ; ScriptInit
20 [-] RETURN 0 1 0
[/code]

You do have to stare at it for a little bit, but it's pretty understandable. Certainly useful for figuring out what functions are being called and what their arguments are.
Using that output, it seems like a lua decompiler would be 'not so hard'. I incorporated this view to a tool I've been using to figure out the .lvl files.

Hidden/Spoiler:

If you'd like to use that utility yourself to understand lua code inside .lvl files, you can try it out for yourself.
Download the SWBF2_Tool.0.51.zip file.
https://github.com/BAD-AL/SWBF2_Xbox_mo ... t/releases

**EDIT**
IT assumes that you have the mod tools located at:
"C:\BF2_ModTools\"

**EDIT**
Updated:
https://github.com/BAD-AL/SWBF2_Xbox_mo ... s/tag/0.52

[Teancum]

Looks very cool, however when doing anything but loading the lvl and populating the search I get exception errors. Here's an example:

Hidden/Spoiler:

See the end of this message for details on invoking
just-in-time (JIT) debugging instead of this dialog box.

************** Exception Text **************
System.ComponentModel.Win32Exception: The system cannot find the file specified
at System.Diagnostics.Process.StartWithCreateProcess(ProcessStartInfo startInfo)
at System.Diagnostics.Process.Start()
at System.Diagnostics.Process.Start(ProcessStartInfo startInfo)
at SWBF2_Tool.ScriptSearchForm.RunCommand(String programName, String args, Boolean includeStdErr)
at SWBF2_Tool.ScriptSearchForm.LuacCodeSize(String luaSourceFile)
at SWBF2_Tool.ScriptSearchForm.ShowCodeSize()
at SWBF2_Tool.ScriptSearchForm.mLuacCodeSizeButton_Click(Object sender, EventArgs e)
at System.Windows.Forms.Control.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnClick(EventArgs e)
at System.Windows.Forms.Button.OnMouseUp(MouseEventArgs mevent)
at System.Windows.Forms.Control.WmMouseUp(Message& m, MouseButtons button, Int32 clicks)
at System.Windows.Forms.Control.WndProc(Message& m)
at System.Windows.Forms.ButtonBase.WndProc(Message& m)
at System.Windows.Forms.Button.WndProc(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.OnMessage(Message& m)
at System.Windows.Forms.Control.ControlNativeWindow.WndProc(Message& m)
at System.Windows.Forms.NativeWindow.Callback(IntPtr hWnd, Int32 msg, IntPtr wparam, IntPtr lparam)


************** Loaded Assemblies **************
mscorlib
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.8825 (WinRelRS3.050727-8800)
CodeBase: file:///C:/Windows/Microsoft.NET/Framework/v2.0.50727/mscorlib.dll
----------------------------------------
SWBF2_Tool
Assembly Version: 1.0.0.0
Win32 Version: 1.0.0.0
CodeBase: file:///D:/Games/SWBF2_Modtools/data_xbx/!!SRC/SWBF2_Tool.exe
----------------------------------------
System.Windows.Forms
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.8825 (WinRelRS3.050727-8800)
CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System.Windows.Forms/2.0.0.0__b77a5c561934e089/System.Windows.Forms.dll
----------------------------------------
System
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.8825 (WinRelRS3.050727-8800)
CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System/2.0.0.0__b77a5c561934e089/System.dll
----------------------------------------
System.Drawing
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.8825 (WinRelRS3.050727-8800)
CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System.Drawing/2.0.0.0__b03f5f7f11d50a3a/System.Drawing.dll
----------------------------------------
System.Configuration
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.8825 (WinRelRS3.050727-8800)
CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System.Configuration/2.0.0.0__b03f5f7f11d50a3a/System.Configuration.dll
----------------------------------------
System.Xml
Assembly Version: 2.0.0.0
Win32 Version: 2.0.50727.8827 (WinRelRS3.050727-8800)
CodeBase: file:///C:/WINDOWS/assembly/GAC_MSIL/System.Xml/2.0.0.0__b77a5c561934e089/System.Xml.dll
----------------------------------------

************** JIT Debugging **************
To enable just-in-time (JIT) debugging, the .config file for this
application or computer (machine.config) must have the
jitDebugging value set in the system.windows.forms section.
The application must also be compiled with debugging
enabled.

For example:

<configuration>
<system.windows.forms jitDebugging="true" />
</configuration>

When JIT debugging is enabled, any unhandled exception
will be sent to the JIT debugger registered on the computer
rather than be handled by this dialog box.

[cbadal]

Good catch.
I guess I never tried listing the assets with no file selected.
Release is updated.

I also disabled some stuff that probably no else (other than me) would use.
It also now populates when you 'Drag N Drop' a file onto the filename text box.

EDIT
Updated:
https://github.com/BAD-AL/SWBF2_Xbox_mo ... s/tag/0.52

[Anakin]

Can't we just use the original scripts, compare them with the compiled once and find this way rules that translate the compiled script back to lua code??

==EDIT==

just tried your decompile lua thing, but seams not to work:

Code: Select all

D:\Star Wars\Tools\SWBF2Tool.0.52>luac
luac: no input files given
usage: luac [options] [filenames].  Available options are:
  -        process stdin
  -l       list
  -o name  output to file `name' (default is "luac.out")
  -p       parse only
  -s       strip debug information
  -v       show version information
  --       stop handling options

D:\Star Wars\Tools\SWBF2Tool.0.52>luac -s ifs_missionselect_pcmulti.script
luac: ifs_missionselect_pcmulti.script:1: invalid control char near `char(16)'

D:\Star Wars\Tools\SWBF2Tool.0.52>

[Teancum]

He is still coding in all the conditional statements, functions, etc. Right now it mostly only works on mission luas but it's doing more with each version.

[Anakin]

I think it was a problem of folder location. I remember that i read somewhere that the folder needs to be in the mod_tools folder. now it worked. That's SOOO awesome.

[cbadal]

Hidden/Spoiler:

[code]-- NAME: addme mLocation: 0x8; Body Length: 2128, Body Start: 0x34, Body End: 0x884
-- C:\BF2_ModTools\space_template\addme\addme.lua
-- PC luac code size = 315; PC code:
--Search through the missionlist to find a map that matches mapName,
--then insert the new flags into said entry.
--Use this when you know the map already exists, but this content patch is just
--adding new gamemodes (otherwise you should just add whole new entries to the missionlist)
function AddNewGameModes(missionList, mapName, newFlags)
for i, mission in missionList do
if mission.mapluafile == mapName then
for flag, value in pairs(newFlags) do
mission[flag] = value
end
end
end
end




--insert totally new maps here:
local sp_n = 0
local mp_n = 0
sp_n = table.getn(sp_missionselect_listbox_contents)

-- INFO 0x1 0x0 0x0 0x0 0x1 0x0 0x0 0x0 [/code]

I noticed that sometimes this tool has a output like this

Code: Select all

-- ********* LUAC Code Size MATCH!!! ***********
-- ********* Binary Equal !!! ***********

does this mean that there is no different to the original script? and is there a way to show only the differences if there are such?

This was posted in the unmunge tool thread, but I thought I'd answer it here.

In it's current state 'SWBF2CodeHelper' is a horrible decompiler. It could not show the differences in source code unless it was a great decompiler.

However, it can show you differences in the code with the 'Listing Compare' feature (You'll need to install 'Beyond Compare' to use that feature though).

Hidden/Spoiler:

Beyond compare is expected to be installed at:
"C:\Program Files\Beyond Compare 4\BCompare.exe" or
"C:\Program Files (x86)\Beyond Compare 4\BCompare.exe"

To use the 'Listing Compare' feature, copy/paste the Luac listing in the left pane; copy/paste the Lua source code into the right pane; then press the 'Compare Listings' button.

Hidden/Spoiler:

This operation will compile the Lua source in the right pane, then get the Luac -l listing from it and compare it to the listing in the left pane.

Hidden/Spoiler:

Using this feature and knowledge of the Lua instruction set will allow someone to decompile the code manually. This is what I used to decompile the PSP mission files that the tool could not decompile on its own.

Lua compiles down to a very simple instruction set (there are only 34 instructions in Lua 5.0) you can gain a very good understanding of these instructions by going through the included guide 'ANoFrillsGuideToLua5VMInstructions.pdf'.
One thing of special note is that the Closure numbers will differ; If only the Closure numbers differ, you should consider the code 'the same'.

Hidden/Spoiler:

[Anakin]

ok, I'll look into that. Btw i got several errors when decompiling all complaining about jmp not defined or something like that. Looks to me that the ircode representation works fine but the compiler does not know what to do with jmp. For some of those error scripts i have the source. If it helps you i can send it to you, so you may can improve the decompiler to work for more (near by all) scripts

[cbadal]

ok, I'll look into that. Btw i got several errors when decompiling all complaining about jmp not defined or something like that. Looks to me that the ircode representation works fine but the compiler does not know what to do with jmp. For some of those error scripts i have the source. If it helps you i can send it to you, so you may can improve the decompiler to work for more (near by all) scripts

'JMP' is a very general instruction. It gets used in different situations. In order to decompile it correctly, you have to do some analysis.
It is used in and/or operations, comparison operations and loops; and it's tricky to figure out just how to decompile it.
If you look at it, you can kinda figure it out. But writing the code that figures it out isn't straight forward and it might never be able to be 100% accurate.

If you have an important script that you're working to decompile, I might be able to help.

My process (from scratch) would be to paste it into SWBF2CodeHelper, try to decompile, if it fails, remove some code, repeat until I have a good base to work with.
Then analyze the listing trying to figure what to splice in and test against until I can do a comparison that results in only the 'CLOSURE' numbers differing.

If it is a modification of an existing file that you are trying to re-construct, then I'd follow the process I mentioned above (pasting code into the right pane, listing into the left pane).

[Anakin]

your tool decompiles this:

Code: Select all

SetupTeams{
             
        rep = {
            team = REP,
            units = 28,
            reinforcements = 150,
            soldier  = { "rep_inf_ep2_rifleman",10, 25},
            assault  = { "rep_inf_ep2_rocketeer",1, 4},
            engineer = { "rep_inf_ep2_engineer",1, 4},
            sniper   = { "rep_inf_ep2_sniper",1, 4},
            officer = {"rep_inf_ep3_officer",1, 4},
            special = { "rep_inf_ep2_jettrooper",1, 4},
            
        },
        cis = {
            team = CIS,
            units = 28,
            reinforcements = 150,
            soldier  = { "cis_inf_rifleman",10, 25},
            assault  = { "cis_inf_rocketeer",1, 4},
            engineer = { "cis_inf_engineer",1, 4},
            sniper   = { "cis_inf_sniper",1, 4},
            officer = {"cis_inf_officer",1, 4},
            special = { "cis_inf_droideka",1, 4},
        }
     }

to this

Code: Select all

  SetupTeams({ 
        rep =         { team = 1, units = 28, reinforcements = 150, 
          soldier =           { "rep_inf_ep2_rifleman", 10, 25 }, 
          assault =           { "rep_inf_ep2_rocketeer", 1, 4 }, 
          engineer =           { "rep_inf_ep2_engineer", 1, 4 }, 
          sniper =           { "rep_inf_ep2_sniper", 1, 4 }, 
          officer =           { "rep_inf_ep3_officer", 1, 4 }, 
          special =           { "rep_inf_ep2_jettrooper", 1, 4 }
         }, 
        cis =         { team = 2, units = 28, reinforcements = 150, 
          soldier =           { "cis_inf_rifleman", 10, 25 }, 
          assault =           { "cis_inf_rocketeer", 1, 4 }, 
          engineer =           { "cis_inf_engineer", 1, 4 }, 
          sniper =           { "cis_inf_sniper", 1, 4 }, 
          officer =           { "cis_inf_officer", 1, 4 }, 
          special =           { "cis_inf_droideka", 1, 4 }
         }
       })

the () is too much

[Teancum]

The game accepts it though.

[cbadal]

Yes,
'SetupTeams' is a function that takes a table. Lua does not require the '()', but it's ok to have it.

[cbadal]

Video Tutorial posted to description.

[Anakin]

Very interesting. When you decompiled the mission, it looks like the original compiled code has no optimization activated. Because all those constants are ignored and the values are used directly. So for some reasons the data flow analysis seems to be deactivate for the compiler frontend when compiling with SWBF2.

Tells me two things: 1) the file size can be reduced much more 2) the decompiling step somewhen applies the data flow analysis. So if you simply deactivate it the reproduction should be much better.