Archive

Archive for the ‘analyzeMFT’ Category

analyzeMFT – ADS support added

The latest version of analyzeMFT is available on github. I’ve not pushed it out to Pypi and will hold off until I’m sure it is free of bugs due to this new work. The changes are:

Fixed parsing and printing of UTF-16 strings, removed unicodeHack stuff.

My original code took a brute force approach to parsing file names from the MFT records. What I did not know at the time was that they were UTF-16. While working on other things, I took the time to figure that out and replaced about 20 lines of kludge with one line of code.

Fixed printing of unicode strings to output files.

While figuring out how to read UTF-16, I figure out how to write UTF-8.

Added ADS support.

This is probably a work in progress but it seems to be working so I’ll push this out. Whenever analyzeMFT encounters a resident $DATA record, it stores a copy of the contents away for later use. If it encounters a named $DATA record, it does two things:

  • A duplicate of the parent record is created and the filename is changed to be <parent filename>:<ADS filename>.
  • All ADS records, parent and children, get a flag set in the new ADS column

So you might see:

/normal.txt Normal file
/file-w-ads.txt Normal file with ADS
/file-w-ads.txt:adsfile.txt The ADS file attached to file-w-ads.txt
/dir Directory
/dir:adsdir.txt The ADS file attached to dir
/file-w-large-ads.txt Normal file with ADS
/file-w-large-ads.txt:largeads.txt The (non-resident) ADS file attached to file-w-large-ads.txt
/file-w-2-ads.txt Normal file with two ADS files
/file-w-2-ads.txt:ads1.txt The first ADS file attached to file-w-2-ads.txt
/file-w-2-ads.txt:ads2.txt The second ADS file attached to file-w-2-ads.txt

All of the records would have a ‘Y’ in the ADS column to indicate that either they are an ADS file or they have an ADS file attached.

As always, please let me know if I broke anything….

Categories: analyzeMFT

Using analyzeMFT from other programs


Now that analyzeMFT is a package, it is much easier to use from other programs. Here’s a quick example.


from analyzemft import mft
input_file = open(‘MFT-short’, ‘rb’)
options = mft.set_default_options()
raw_record = input_file.read(1024)
mft_record = {}
mft_record = mft.parse_record(raw_record, options)
print “\nRaw MFT record in analyzeMFT format”
print mft_record
csv_record = mft.mft_to_csv(mft_record, False)
print “\nMFT record in CSV format”
print csv_record
l2t_record = mft.mft_to_l2t(mft_record)
print “\nMFT record in L2T format”
print l2t_record
body_record = mft.mft_to_body(mft_record, options.bodyfull, options.bodystd)
print “\nMFT record in bodyfile format”
print body_record

This will produce:


Raw MFT record in analyzeMFT format

{‘f1’: ‘\x00\x00’, ‘seq’: 1, ‘lsn’: 4.365328012e-314, ‘attr_off’: 56, ‘bitmap’: True, ‘alloc_sizef’: 1024, ‘recordnum’: 0, ‘size’: 424, ‘upd_off’: 48, ‘filename’: ”, ‘upd_cnt’: 3, ‘base_seq’: 0, ‘fncnt’: 1, ‘link’: 1, ‘next_attrid’: 6, ‘data’: True, ‘base_ref’: 0, ‘magic’: 1162627398, (‘fn’, 0): {‘par_ref’: 5, ‘ctime’: <analyzemft.mftutils.WindowsTime instance at 0x107864d40>, ‘par_seq’: 5, ‘nlen’: 4, ‘flags’: 3e-323, ‘real_fsize’: 32686080, ‘mtime’: <analyzemft.mftutils.WindowsTime instance at 0x107864830>, ‘alloc_fsize’: 32686080, ‘nspace’: 3, ‘atime’: <analyzemft.mftutils.WindowsTime instance at 0x1078649e0>, ‘crtime’: <analyzemft.mftutils.WindowsTime instance at 0x107864368>, ‘name’: ‘$MFT’}, ‘notes’: ”, ‘si’: {‘maxver’: 0, ‘ver’: 0, ‘ctime’: <analyzemft.mftutils.WindowsTime instance at 0x1078648c0>, ‘class_id’: 0, ‘usn’: 0.0, ‘sec_id’: 256, ‘quota’: 0.0, ‘own_id’: 0, ‘mtime’: <analyzemft.mftutils.WindowsTime instance at 0x1078647e8>, ‘dos’: 6, ‘atime’: <analyzemft.mftutils.WindowsTime instance at 0x1078645a8>, ‘crtime’: <analyzemft.mftutils.WindowsTime instance at 0x10777ac20>}, ‘flags’: 1}

MFT record in CSV format

[0, ‘Good’, ‘Active’, ‘File’, ‘1’, ‘5’, ‘5’, ”, ‘2007-08-15 15:32:29.656248’, ‘2007-08-15 15:32:29.656248’, ‘2007-08-15 15:32:29.656248’, ‘2007-08-15 15:32:29.656248’, ‘2007-08-15 15:32:29.656248’, ‘2007-08-15 15:32:29.656248’, ‘2007-08-15 15:32:29.656248’, ‘2007-08-15 15:32:29.656248’, ”, ”, ”, ”, ”, ”, ”, ”, ”, ”, ”, ”, ”, ”, ”, ”, ”, ”, ”, ‘True’, ‘False’, ‘True’, ‘False’, ‘False’, ‘False’, ‘True’, ‘False’, ‘False’, ‘True’, ‘False’, ‘False’, ‘False’, ‘False’, ‘False’, ”, ‘N’, ‘N’]

MFT record in L2T format

2007-08-15|15:32:29.656248|TZ|…B|FILE|NTFS $MFT|$FN […B] time|user|host||desc|version||1||format|extra

MFT record in bodyfile format

0|$MFT|0|0|0|0|32686080|1187191949|1187191949|1187191949|1187191949

 


Simple. Hand it a raw MFT record and then ask for the results to be produced in a string in one of three formats. (Hmm, I suppose I should support JSON, too.)

Categories: analyzeMFT

analyzeMFT now available via pip

[Ed Note: Please excuse the formatting. WordPress seems to be doing something funky.]

analyzeMFT just got two major, and related upgrades:

  • You can install it via PyPi
  • It is now a well behaved (?) package and can more easily be included in other programs.

PyPi:

pip install analyzeMFT

Alternatively:

git pull https://github.com/dkovar/analyzeMFT.git
python setup.py install

or, just run it from that directory.

The main program is now much simpler:

#!/usr/bin/python
try:
 from analyzemft import mftsession
except:
 from .analyzemft import mftsession

if __name__=="__main__":
session = mftsession.MftSession()
session.mft_options()
session.open_files()
session.process_mft_file()
session.print_records()

The main program just opens a session, gets options, opens the files, processes the records, and prints the results. All of the records are available via:

session.mft[seqnum]

Where seqnum is the sequence number of the record you want to reference.

You should also be able to ask it to process a single record and return it in raw, bodyfile, L2T CSV, or normal CSV form. If this would be useful, let me know and I’ll document and confirm the process.

Categories: analyzeMFT

First steps in converting analyzeMFT to a Python module, plus improved error handling

I started rewriting analyzeMFT so that it can be loaded as a module and called from other programs. The primary reason is to enable including it in plaso, but perhaps other programs will find a need for it.

The work isn’t done yet, but it is usable as a standalone program still and it has some improved handling of corrupt MFT records so I decided to release it.

Quick install:

Once I finish the work I’ll also make a zip file available.

Notes:

  1. All output between the new and old version is identical except in cases where records are corrupt or incomplete. In those cases, the new output is more accurate.
  2. There is a lot of strangeness going on in MFT records. In restructuring analyzeMFT, I found a number of conditions that I failed to check for but which accidentally didn’t throw errors. For example, there are MFT records with no Standard Information attributes.
  3. Detection of Orphan records, my term, has been improved. Additional research is required to determine what causes them to occur.
  4. Processing time improved slightly
Categories: analyzeMFT

Improved bodyfile support

April 26, 2013 3 comments

With more thanks to Jamie for the prompting, I’ve improved bodyfile support in the latest version of analyzeMFT.

  • You can now specify just a bodyfile for output and do not need to create a normal output file as well.
  • The real (not allocated) file size is included
  • If you use the –bodypath option, it writes out the full path to the file rather than just the file name
  • If you use the –bodystd option, it uses the STD_INFO timestamps rather than just the FN timestamps. I find STD_INFO to be more interesting….

This is a pretty significant fix and I would suggest upgrading if you create timelines with analyzeMFT.

Links:

Git: git clone https://github.com/dkovar/analyzeMFT.git
Code: https://github.com/dkovar/analyzeMFT/blob/master/analyzeMFT.py

Categories: analyzeMFT

Updated analyzeMFT – fixed MFT record number reporting

When I originally wrote analyzeMFT I assumed that the MFT record numbers would start at zero and politely increase by one for each record so “recordNumber = recordNumber + 1” would be valid. Happily, this worked, apparently for years. That is, until Jamie threw corrupted MFT files at it, such as MFT records extracted from memory.

  1. The sequence numbers had gaps
  2. If there was a gap, then the actual sequence number wouldn’t match the reported sequence number
  3. Determination of the file path might be off as the parent record number pulled from the entry might now point to the wrong entry

Oooops.

This has been fixed.

I also fixed the handling of orphan files, those files that had a null parent or whose parent was a file.

This is a pretty significant fix and I would suggest upgrading.

Links:

Git: git clone https://github.com/dkovar/analyzeMFT.git
Code: https://github.com/dkovar/analyzeMFT/blob/master/analyzeMFT.py

Categories: analyzeMFT

analyzeMFT has moved to GitHub

Just a quick note to say that analyzeMFT has moved to GitHub:

https://github.com/dkovar/analyzeMFT

I’ve got some other things in the works and was looking for a place that would allow me to neatly consolidate them all. The fact that GitHub allows for private and public repos in one account was a big selling point, but there are other factors.

Categories: analyzeMFT