Msh chunk interface

Releasing the source files for your mod or map? Post em' here. (Applies to both SWBF1 & SWBF2)

Moderator: Moderators

User avatar
giftheck
Droid Pilot Assassin
Droid Pilot Assassin
Posts: 2218
Joined: Mon Jan 19, 2009 5:58 pm
Projects :: Star Wars Battlefront Anniversary

Re: Msh chunk interface

Post by giftheck »

Anakin wrote:
ggctuk wrote:
AceMastermind wrote:
ggctuk wrote:What I mean is, does it prevent model glitches like these?
Hidden/Spoiler:
Image
Image
Image
Those aren't glitches, they look like user errors in weighting prior to export. This can be fixed in 5 minutes within XSI.
They are all hex-edited models - the last I hex-edited myself using the clean Ep3 model. Maybe there's something about the import process that messes the weighting up.

jep this was an error in XSI. i deletet an edge XD
No, it's actually caused by something I did in hex-editing. It seems I somehow made the left foot bone corrupted in the hierarchy. Maybe hex-editing too many addons does that, eh?
User avatar
DarthD.U.C.K.
Master of the Force
Master of the Force
Posts: 6027
Joined: Wed Sep 27, 2006 11:05 am
Location: Duckburg, Germany

Re: Msh chunk interface

Post by DarthD.U.C.K. »

the thing is (from what i understand) that there is nothing like "addons" within a msh, once you hexedited a modlchunk in, its not different from any modl chunk that was originally in the msh.
User avatar
giftheck
Droid Pilot Assassin
Droid Pilot Assassin
Posts: 2218
Joined: Mon Jan 19, 2009 5:58 pm
Projects :: Star Wars Battlefront Anniversary

Re: Msh chunk interface

Post by giftheck »

DarthD.U.C.K. wrote:the thing is (from what i understand) that there is nothing like "addons" within a msh, once you hexedited a modlchunk in, its not different from any modl chunk that was originally in the msh.
I know that. What I'm talking about, is that it must have been something I did hex-editing the model chunk into the MSH file.

Maybe it's something that can be fixed with this tool since it does the hard part for you.
madk
Recruit Womprat Killer
Posts: 10
Joined: Fri Nov 18, 2011 7:44 pm
Projects :: Procrastinating on my NaNoWriMo
Games I'm Playing :: Oblivion and SWBFII

Re: Msh chunk interface

Post by madk »

Thought I'd drop in that I'm currently working on features that allow you to work with the actual data within the file if you so wish. (So you can change things like texture file names, and colors, and stuff, provided you know where to find them. I might implement more features that find them for you if I think it's worthwhile.)
User avatar
ANDEWEGET
Ancient Force
Ancient Force
Posts: 1266
Joined: Tue Apr 01, 2008 8:42 am
Location: Germany
Contact:

Re: Msh chunk interface

Post by ANDEWEGET »

@madk:
Adding the ability to scroll with the mouse wheel and maybe a "tree" view(where youre able to expand/close chunks for better overview) for the chunk box would very much improve the usability/efficiency. And making the window and the widgets a bit larger wouldnt hurt either. Its not like youre developing this for a smartphone. ;)

@tirpider:
I read through your last post over at swbfgamers. The idea of removing parts from a model via the msh file is pretty straightforward. The actual code would be complex and long I believe because you have to change/unpack/repack/maybe even move values at quite a lot of places.

Something like this I suppose:
1. Remove(not zero, remove completely) all points you dont need/want(maybe by checking if theyre under a certain Y value).
2. Remove values in STRP which refer to the points you removed(you most likely have to unpack them first for that, dunno how ZE handles problems with polygons but you might have to check if every poly has 3 points).
3. Remove UVs of the removed points(AFAIK uvs refer to the points at the same position in the POSL chunk).
(4. Remove normals, dunno which exactly or if you have to)
(5. Remove points from WGHT)
(6. Remove values from NDXT/NDXL? If its true that they arent used anymore then ignore them.)
7. Change the length values after the chunk headers(POSL, UV0L, STRP, NRML, SEGM, GEOM, MODL, MSH2, HEDR) accordingly.
madk
Recruit Womprat Killer
Posts: 10
Joined: Fri Nov 18, 2011 7:44 pm
Projects :: Procrastinating on my NaNoWriMo
Games I'm Playing :: Oblivion and SWBFII

Re: Msh chunk interface

Post by madk »

>scroll with the mouse wheel
I was thinking I should do that. I'll be sure to include it

>a "tree" view for the chunk box
already added, just want to clean up the rest of the half-implemented features before I release it

>And making the window and the widgets a bit larger wouldnt hurt either.
I will keep that in mind then :P
madk
Recruit Womprat Killer
Posts: 10
Joined: Fri Nov 18, 2011 7:44 pm
Projects :: Procrastinating on my NaNoWriMo
Games I'm Playing :: Oblivion and SWBFII

Re: Msh chunk interface

Post by madk »

Download updated.

Image

changes:
scroll wheel!
tree view
edit chunk's data, the program tracks all the related numbers that need to change with it so the file doesn't break (you can disable this automatic handling if you want, but I definitely would not recommend it)
you can't input floating point numbers just yet, (edit: now you can) it's going to be a bit of work to implement and I want to get this out. however, integers and null-terminated strings work fine.
a couple graphical bugfixes
User avatar
tirpider
Lance Corporal
Posts: 97
Joined: Fri Nov 25, 2011 3:58 pm

Re: Msh chunk interface

Post by tirpider »

Looks great, madk.
I dig the MODL hierarchy on the right.

I found a way to handle 32 bit floats with AutoIT.
(_WinAPI_IntToFloat) It translates a hex string to a float, even though it's name is misleading.
Since it's WinAPI, there is probably a simular way to get at it from whatever language your using.

Getting it back into a hex string is a pain, though. Right now I'm using a DLLStruct :? and someone else's code from AutoIT's forum. I don't understand how it works, but it does.

All my efforts at finding the actual math involved have led to proprietary, 'black box' solutions where some function, or someones web app, solves it for you.

@ANDEWEGET
I am working on exactly what you suggested, but keeping all the contingent edits lined up is slowing me down.

Thanks for the heads up on NDXT and NDXL, I tried out a few models with them removed and BF1 and ModelMunge.exe don't seem to notice. No errors or noticable diff in game. They do seem a bit redundant since STRP already has triangle info and is more efficiant anyway.


(on a side note, perhaps this post wont be declined like the last one.)
User avatar
ANDEWEGET
Ancient Force
Ancient Force
Posts: 1266
Joined: Tue Apr 01, 2008 8:42 am
Location: Germany
Contact:

Re: Msh chunk interface

Post by ANDEWEGET »

Good to know. ;) I like me my python, floatVar = struct.pack('<f', 0.9) is all I need to convert a float to binary... :P

@madk:
What happens if your tool reads a corrupt msh file? Is there some sort of popup or something? Could come in handy if I could use it to validate msh files.
madk
Recruit Womprat Killer
Posts: 10
Joined: Fri Nov 18, 2011 7:44 pm
Projects :: Procrastinating on my NaNoWriMo
Games I'm Playing :: Oblivion and SWBFII

Re: Msh chunk interface

Post by madk »

It doesn't currently, but I'll add a "verify" button or something for you



edit: made (and, augh, did it take a lot of work including analysis of some largely undocumented chunks) but I likely won't be able to update the dropbox link until monday. if issues were found it spits 'em out into a log file so you can see precisely what's up.
know that if you go and change some random byte you will probably screw up, or at least mangle, how the game sees the mesh, but the program isn't going to spot random problems like that. while it should be very reliable for most applications, don't expect it to be 100% accurate.

sometime later I may put down a summary for all the stuff I've had to figure out that hasn't been documented anyplace I've looked off of for the file structure.

EDIT
the download has been updated. now it has a "verify" button that can tell you, with a reasonable degree of accuracy, whether the msh is going to work ingame

Double posting is against the RULES; please EDIT your post instead -Staff
User avatar
Teancum
Jedi Admin
Jedi Admin
Posts: 11080
Joined: Wed Sep 07, 2005 11:42 pm
Projects :: No Mod project currently.
Games I'm Playing :: Destiny
xbox live or psn: No gamertag set
Location: Indiana

Re: Msh chunk interface

Post by Teancum »

Just curious - this works with the "head swap" trick, too? Modeling was never my forte.
User avatar
skelltor
Sith
Sith
Posts: 1431
Joined: Fri Dec 11, 2009 6:24 pm
Projects :: BFSM
Games I'm Playing :: SWBFII
xbox live or psn: skelltor95
Location: minisnowta
Contact:

Re: Msh chunk interface

Post by skelltor »

I am a bit confused does it do textures now too?
AQT
Gametoast Staff
Gametoast Staff
Posts: 4910
Joined: Sat Nov 03, 2007 4:55 pm
Location: SoCal, USA

Re: Msh chunk interface

Post by AQT »

[b][color=#FF8000]madk[/color][/b] wrote:If something can't be done with hex editing it probably can't, and likely won't be possible to, be done with this.
@Teancum: It depends on the head. If the head is a seperate MODL or SEGM chunk, then it can be removed. If the head is a seperate MODL chunk, then it can be transferred. If transferring the new head to a model whose original head was a SEGM, then there could be problems getting the entire model to work, especially if both heads had an ENVL chunk.
skelltor wrote:I am a bit confused does it do textures now too?
madk wrote:I'm actually working on a bit of code right now to recognize and let the user handle matl chunks. Matl chunks are done slightly differently from the others but the exception should be rather trivial to implement.
He actually meant to write MATD chunks, but it seems this tool isn't capable yet of transfering the chunks that define the textures.
User avatar
ANDEWEGET
Ancient Force
Ancient Force
Posts: 1266
Joined: Tue Apr 01, 2008 8:42 am
Location: Germany
Contact:

Re: Msh chunk interface

Post by ANDEWEGET »

Played around a bit with it...
Your tool considers CLRB chunks to be illegal(they store info about Vertex Colors I believe).
Sometimes it seems to duplicate materials.
It crashes when it tries to load a .msh with wrong length indicators(popping up a window which tells me that the msh file is corrupt would be nice :), maybe with some info about at which offset or chunk it crashed).
Good work though! I really like the Verify function!
User avatar
tirpider
Lance Corporal
Posts: 97
Joined: Fri Nov 25, 2011 3:58 pm

Re: Msh chunk interface

Post by tirpider »

In order to import a MODL, your gonna need to re-index the MNDX's anyway.
While doing that, go ahead and map all the ENVL's out.
While loading and parsing the tags of the new MODL chunk, rewrite it's MNDX and it's ENVL (then update the other ENVL's already in the msh.)
madk
Recruit Womprat Killer
Posts: 10
Joined: Fri Nov 18, 2011 7:44 pm
Projects :: Procrastinating on my NaNoWriMo
Games I'm Playing :: Oblivion and SWBFII

Re: Msh chunk interface

Post by madk »

ANDEWEGET wrote:Played around a bit with it...
Your tool considers CLRB chunks to be illegal(they store info about Vertex Colors I believe).
Sometimes it seems to duplicate materials.
It crashes when it tries to load a .msh with wrong length indicators(popping up a window which tells me that the msh file is corrupt would be nice :), maybe with some info about at which offset or chunk it crashed).
Good work though! I really like the Verify function!
where exactly do CLRB chunks belong?

and I'll do something about the length indicator issues.

tirpider wrote:In order to import a MODL, your gonna need to re-index the MNDX's anyway.
While doing that, go ahead and map all the ENVL's out.
While loading and parsing the tags of the new MODL chunk, rewrite it's MNDX and it's ENVL (then update the other ENVL's already in the msh.)
from what I understand, MNDX's are insignificant. and I honestly have no idea what you mean by
mapping out ENVL's.

regardless, I would rather leave the MNDX editing to the user. I don't want to clutter the tool with functions that can be easily performed with a little bit of user competence. (all it takes is selecting the MNDX chunk and inputting a new index)
User avatar
ANDEWEGET
Ancient Force
Ancient Force
Posts: 1266
Joined: Tue Apr 01, 2008 8:42 am
Location: Germany
Contact:

Re: Msh chunk interface

Post by ANDEWEGET »

CLRB is a child chunk of SEGM. I dunno how exactly it works though, its just one 32-bit int(or maybe 4 8-bit ints...). Either .msh files only support one color per SEGM(this could fit, one 8-bit int for every channel, RGBA)and it stores the color or it tells modelmunge that certain vertexes have a certain color but the color isnt stored anywhere so...
User avatar
tirpider
Lance Corporal
Posts: 97
Joined: Fri Nov 25, 2011 3:58 pm

Re: Msh chunk interface

Post by tirpider »

Re: Mapping The entries in ENVL.

The ENVL entries are a direct enumeration of the MNDX's of the MODL elements that make up the envelope as is relevant to the MODL the ENVL is in. (ie, the head MODL would only have entries for bone_head and perhaps neck..) All the MODL entries referenced by ENVL have an MTYP value of 3 (meaning it's an envelope piece [Riley pages].) Typicaly named bone_* , but not always.

Inserting a MODL chunk without altering it's MNDX can be done without messing things up to much if it is properly ordered in the heiarchy of chunks, but If it's MNDX is the same as something important, say, the bone_root, or on the other end, bone_l_toe, every ENVL that uses those could end up refering to an incorrect MODL.

If the MNDX is insignificant, then the numbers in ENVL must refer to the MODL's by order of appearance (identical to MNDX btw.) and would be offset by the introduction of any MODL chunk anywhere but at the very end.

I was just suggesting that proactively setting the ENVL chunks and MNDX entries could shave a lot of time off the would-be model hacker's work time, as the changes made would be dictated by the msh structure itself and not a creative one.

-edit-
urg. Reflecting on that, perhaps it's not so easy to do, by hand or to automate.
User avatar
Maveritchell
Jedi Admin
Jedi Admin
Posts: 7366
Joined: Mon Aug 21, 2006 11:03 pm

Re: Msh chunk interface

Post by Maveritchell »

Just from a user perspective, it doesn't appear as though your tool fully supports meshes exported using RepSharpshooter's MshEx tool. It doesn't look like the MODL chunks contained in those are being completely parsed - I know that RepSharpshooter stripped some "superfluous" data from these that allows them to be decompiled with some tools (for example, his own meshtool has had problems converting them to an .obj format).
meapineapple
Posts: 2
Joined: Wed Mar 01, 2023 12:30 pm
Projects :: No Mod project currently.
xbox live or psn: No gamertag set

Re: Msh chunk interface

Post by meapineapple »

Please forgive me for necroposting in this thread! I am the original author of this mshedit tool, using yet another forum account. I have noticed that this is listed as a resource in the Asset Links Thread but there is no longer any working download link for this tool.

Here is a new link. I would like to encourage anyone who might be concerned with the preservation of this tool to rehost it elsewhere and share the link here.

https://files.pineapplemachine.com/publ ... shedit.zip
Post Reply