Hi, Minions:
This is another
article, one of those that, for one reason or another, 'choke' for various
reasons. I had it pending, on my 'ToDo' list, half completed, for a long, long
time. And it's for that reason, because a lot of time has passed, that I've
restarted all those tests that I had half completed. Because the working
environment changes, because the systems change.
You could say that
this article came up by chance, a very long time ago... although it has some
relation with another article that I took up, also, a long time ago, and that I
published under the title of 'One
byte makes the difference: MFT Resident File'.
Introduction
We must know and
understand how the systems we are analysing behave. There are a multitude of
characteristics in them. These characteristics may be known or may yet be
'discovered'. Or, even, it may be the case that they are presented in some
versions and not in others, as it happens, for example, with the case of the
content I published in the article 'Windows
and its anti-forensic side, (Plug and Play Cleanup and setupapi[.]dev)'.
Some of those features may be recent or may have been there all your life. Some
of these features may have a strong impact on a forensic analysis and some of
these features may not have a serious impact on certain types of analysis. But that's something we won't get to know if we
can't understand how those features work in systems. It is important to
have a deep understanding of systems, as Harlan Carvey mentioned in an
article he published on May 8, 2019, (Deep
Knowledge, and the Pursuit Thereof).
There are times
when we are so focused on a task that, despite observing strange behaviour in
the work environment, we do not pay the
attention it deserves because, apparently, it has nothing to do with the
task we are carrying out at the time.
It's happened to
me, and it's going to happen to me. And, surely, it has happened to you and it
will happen to you.
Big mistake.
The feature I
want to talk to you about today is the tunnelling
of the file system. You're probably familiar with this feature, or maybe
you're not familiar with it at all. In any case, I'm sure you've seen it on
more than one occasion.
"It's important to have
a thorough understanding of systems, but it's impossible to know everything
about them.”
So, as Brett
Shavers rightly says in the article he published under the title "Don't
look back. Try to keep up. This is #DFIR."
"When you search for an
answer and find it, retaining what you learn is much better than posting a
question in a forum and waiting for someone to provide the answer."
The importance
of documentation
It is vital to document yourself before starting a
job.
File system tunneling is considered by some analysts to be a "somewhat obscure" feature of
Windows. This feature is that when
you delete a file and create another one, with the same name and in the same
path, the new file created inherits the same creation date as the file that has
been deleted. This happens if it is done within a certain time, (according to some articles I have been
reading, that time can be up to 15 seconds, by default). That is, you
have to give some specific conditions
to be able to observe how this functionality
reuses some metadata from another file.
"If we don't know these
'special' characteristics, we can get confused."
By not knowing
this type of characteristics, we fall into confusion for not being able to give
an answer, for not being able to explain, the reason for this peculiarity.
I've read and
studied the articles I've found on this subject. Let's do some chronology!
Some very
interesting articles have been written on this topic, although the original Microsoft documentation
that mentions this feature, and that those other articles reference, is no longer available, (http://support.microsoft.com/?kbid=172190).
(I asked the Infosec and DFIR
community, but no one seems to have the content of that article, since no one has ever responded to me). So,
right now, there is no official documentation about it. I hate that official
documentation is missing.
"Lack of documentation
increases confusion."
Luckily, Dan O'Day was
kind enough to keep some of the content of some of those 'lost' items in his GitHub
repository.
It seems that
this article from Microsoft mentioned that both NTFS and FAT file systems have
this peculiarity to create tunnels.
I quote from the filesystem tunneling content that Dan copied into his
repository and which has been mentioned by other articles as well:
"The Microsoft Windows
products listed at the beginning of this article contain file system tunneling
capabilities to allow compatibility with programs that rely on file systems to
maintain file meta-information for a short period of time.
"When a file is deleted
from a directory, (rename or delete), its short/long name pair and the time of
creation are saved in a cache, along with the name that has been deleted. When
a file is added to a directory, (rename or create), the cache is searched for
information to restore. The cache is effective for each instance of a given
directory. If a directory is deleted, the corresponding cache is removed."
"The idea is to mimic
the behavior that MS-DOS programs expect when they use the secure storage
method. They copy the changed data to a temporary file, delete the original and
rename the temporary to the original. This should look like the original file
when it is complete. Windows tunnels through the FAT and NTFS file systems to
ensure that the long/short file names are preserved when 16-bit applications
perform this secure storage operation.”
"The tunnel cache time
can be adjusted from the default time of 15 seconds, or if the tunnel
capabilities are not desirable, it can be turned off by adding a value in the
Windows Registry.”
"If tunneling is
disabled, applications using this secure storage method may lose the name they
do not know, usually LFN, and rediscovery of the shortcut targets could be
affected, as the creation timestamps cannot remain constant for files handled
by such applications. Maintaining the creation timestamp is possible in the
absence of tunneling if an application is smart enough. The same is not true
for long file names."
"...allow compatibility
with programs that rely on file systems to maintain file meta-information for a
short period of time. This occurs after deleting or renaming and recreating a
new directory entry with that meta-information, (if a creation or renaming occurs
that causes a file with that name to appear again in a short period of
time)".
……….
File system metadata is always cached, as you can read in this article
from Microsoft's official site.
In this other
article by Microsoft, dated July 15, 2005, (The apocryphal
history of file system tunnelling), which already mentions the
creation of tunnels in file systems, the reason for tunneling is explained,
mentioning that this feature was already
implemented in Windows 95.
I can't help but
get excited... So much nostalgia... Windows 95...
This article
also mentions that after deleting a file
with a long file name, the file generated later, within a certain time, with
the same name and identical location, will inherit, in addition to the creation
date, the same short
file name as the deleted file.
The last
sentence of that article caught my attention very much:
"Fragments of
information about recently deleted or renamed files are stored in data structures
called 'quarks'.”
That is,
according to this Microsoft article, during
the tunneling effect some of the information in the affected file is
temporarily saved.
Looking for
information about this, I found an article published on August 22, 2007, (Windows
Date Created Timestamp strangeness), which says that Windows caches the timestamp of the deleted
file for some time, and that this time can be up to 8 minutes.
On April 11,
2010, Harlan already started to publish material related to this
file system tunneling functionality. As mentioned by Harlan in that article,
under normal circumstances, timestamps,
(via timelines), already pose some
difficulty in analysis. If you add to that difficulty more elements that
are perhaps unknown... 'Turn off and let's go!
It is mentioned
in another article, published on 7 February 2012, (File
System Tunneling in Windows), that tunneling is related to the fact
that some programs using the secure
storage method copy the modified data to a temporary file before deleting the
original file and rename the temporary file with the original file name.
File system tunneling can be disabled and can also be
assigned a specific time, given in seconds. The first article I've seen that mentions this,
besides the extract from the official documentation that Dan kept, is one that
was published on February 24, 2014, (Microsoft
Windows File System Tunneling). This article says that these two
operations can be carried out by adding a key pair to the Windows Registry:
To disable the file system tunneling,
simply add the key
“HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem\MaximumTunnelEntries”
And assign that
key a value of '0'.
To modify the
time of the tunneling it would be enough to add the key
“HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem\MaximumTunnelEntryAgeInSeconds”
And assign that
key a set value in seconds.
Harlan also
published another very interesting article on this subject, with interesting
evidence, on July 22, 2014, (File
system ops, effects on MFT records). According to the evidence he
presented, he concluded that during the
file system tunneling, the original file record in the MFT is marked as
'deleted' and that a new record is assigned to the new file created during that
tunneling. You could also see that $UsnJrnl
contains entries showing that a file system tunneling has taken place.
On August 2,
2013, Blazer
Catzen published in David Cowen's Blog a good
presentation on this topic, which you can
download from his Google site.
I couldn't find
any other material that caught my attention until Hideaki Ihara first published an
article on October 12, 2018, (File System
Tunneling and E:\), where he performs some tunneling tests on a USB
device. Hideaki published another article, on October 19, 2018, (File System
Tunneling and C:\), performing the same tests on a local disk.
Finally, more
recently I found another interesting article, published on April 13, 2019, (File
System Tunneling in Windows), which is very well explained, with
interesting proofs, and which links to a very interesting PDF document, which
you can download
from Microsoft's official site. This document, titled "File
System Behavior in the Microsoft Windows Environment", mentions, among
other things, that FAT, NTFS and exFAT file systems support file tunnelling,
while the UDF file system does not support file tunnelling.
And this is the
extent of the documentation I have followed in order to study and understand
tunneling. But... Even with all that has been written, I have some doubts.
Doubts that I'm going to try to clear up on my own.
The test environment
For the tests I
have carried out, I have chosen to use a Windows
10 system, in its version 18363,
and a Windows 10 system, in its version 17763. Both systems have been
virtualized with VirtualBox,
assigning 8 GB of RAM and 4 processors to each of them.
You can download
Microsoft Windows virtual machines for virtualized environments from their
official site at the following links:
- https://developer.microsoft.com/en-us/microsoft-edge/tools/vms/
- https://developer.microsoft.com/es-es/windows/downloads/virtual-machines/
On each of the
virtualized Windows 10 systems I performed a number of actions. More
specifically, I can say that I divided the tests into the following five actions
for each system:
- CCDR - Create a file 'A' in a folder; create a file
'B', in the same folder; delete file 'A'; and rename file 'B' as 'A'.
- CCRR - Create a file 'A' in a folder; create a file
'B', in the same folder; rename file 'A' as 'C'; and rename file 'B' as 'A
- CDC - Create a file 'A' in a folder; delete file
'A'; and recreate a file 'A', in the same folder
- CMC - Create an 'A' file in a folder; move the 'A'
file to another folder; and recreate an 'A' file, in the same folder.
- CRC - Create a file 'A' in a folder; rename file 'A'
as 'B'; and create a file 'A', in the same folder.
The meaning of
the abbreviations is:
- C = Create
- D = Delete
- R = Rename
- M = Move
In turn, each of
these actions has been carried out under a series of conditions, combined with
each other:
- A waiting time
of 1400 milliseconds.
- A waiting time
of 1500 milliseconds.
- A waiting time
of 1600 milliseconds.
- Long file names.
- Short file names.
- A file size of
512 bytes.
- A file size of
4096 bytes.
At the same
time, all these tests have been executed:
- On a system
with a default configuration.
- On a system
with tunneling disabled.
- On a system
with a certain tunneling value.
- On a system
with a maximum tunneling value.
- In a system
with the creation of short names
disabled.
- In a system
with timestamp updating disabled.
These are just
some of the possible combinations I've come up with, but there's more. Even so,
you can calculate the total number of runs I have carried out, on both systems,
and get an idea of the time spent on all these tests, and their subsequent
analysis. You can rest assured, because
I will not expose all the results I have obtained in the tests. I'll just stick
to those that I think are of interest. But don't miss out on a good cup of
coffee ;)
Each of the actions, described above, has been
automated by a small script in PowerShell, with the following structure
During the
execution of each one of these scripts I have proceeded to extract the files '$MFT' and '$LogFile', with the OSForensics
tool, in its portable version, (You can use any other, like FTK
Imager Lite). The extraction of these files has been carried out
just after the creation of the first file and just after the execution of the
script. That is, both elements have been extracted twice, for each execution,
in order to compare the before and after
information.
Similarly, I have
allowed a total of 300 seconds to elapse, just after preparing the files for
the tunnel effect. 300 seconds is more than enough time for the extraction of
the necessary elements, for its later analysis, and to carry out some
screenshots with the tool Active@ Disk Editor,
in order to study the different attributes and metadata of each one of the
treated files.
Regarding the
processing of the extracted '$MFT' and '$LogFile' files, I have chosen to use
the FTE
and MFTECmd tools for
the analysis of the '$MFT', (for the
contrasting data and other weird things I like to do), and the LogFileParser
tool for the analysis of the '$LogFile'.
A short version...
Thank you, statistics!! I have been able to verify, thanks to the statistics,
that you will not spend more than two minutes reading any of my articles. That's
why I'm going to present now, and not at the end, as I usually do, a brief
summary of the work done.
The file system
tunneling will become visible, according
to my tests, if the actions are carried out within 15000 milliseconds,
on a system with a default configuration. All this, regardless of the size of
the file and the type of file name, with one exception.
Tunneling is
generated when:
- A file 'A' is
created in the folder 'X', a file 'B' is created in the folder 'X', file 'A' is
deleted and file 'B' is renamed to 'A'.
- A file 'A' is
created in the folder 'X', a file 'B' is created in the folder 'X', file 'A' is
renamed as 'C' and file 'B' is renamed as 'A'.
- A file 'A' is
created in the folder 'X', file 'A' is deleted, another file 'A' is created in
the folder 'X'.
- A file 'A' is
created in the folder 'X', file 'A' is moved to the folder 'Z' and another file
'A' is created in the folder 'X'.
- A file 'A' is
created in the folder 'X', the file 'A' is renamed as 'B' and another file 'A'
is created in the folder 'X'.
Of the cases
mentioned above, the only exception where the tunnel effect will not be
generated is in the case of combining a long file name with its short name
equivalent.
If file system
tunneling is disabled, it will not occur under any conditions, except for the
very action of overwriting a file.
You can
customize the file system tunneling time, up to a maximum, determined in
seconds.
The maximum time
the file system can be tunnelled is 71000 milliseconds.
Disabling the
creation of short file names, (8.3), also disables the tunnelling of the file
system, since tunnelling is based on such names.
Disabling the
update of timestamps does not affect the tunnel effect. That is, even if this
function is disabled, it is still generated.
Not in all cases
a new entry is assigned in the MFT for the file causing the tunnel effect. In
fact, in those actions that imply the elimination of file 'A', the entry in the
MFT is reused, for file 'B'. This has been seen by disabling the file system
tunneling, by disabling the creation of short names, (8.3), and in the case of
file overwriting.
We will not
always find, when a tunnel effect is generated, a discordance in the creation
dates of the attributes 'SI' and 'FN'. In fact, there will be occasions when we
must base the study on the discordance of dates within the 'SI' attribute.
The answer to
the explanation of a tunnel effect will be found in the analysis of the
'$LogFile' file.
Testing in a Default System
CCDR
Create a file 'A' in a folder; create a file 'B', in
the same folder; delete file 'A'; and rename file 'B' as 'A
- Create a 512-byte 'A' file with a long name
- Create a 512-byte 'B' file with a long name
- Delete the 'A' file
- Set a timeout of 14000 milliseconds
- Rename file 'B' to 'A
In this test, as
you can see in the previous animated image, a file with the long name
'512-14-LFN01' was created in the system in the 'Test' folder, on 29-05-2020 at
16:12:45. This file was assigned the entry number 93994 in the MFT. After
waiting 10 seconds, a second file was created with the long name
'512-14-LFN02', in the same folder, at 16:12:55. This second file was assigned
entry number 93996 in the MFT.
After waiting
300 seconds, the file was deleted with the name '512-14-LFN01' and, after a
wait of 14000 milliseconds, the file was renamed '512-14-LFN02' to
'512-14-LFN01'.
As you can see
from the above table extracts, the file '512-14-LFN01', which occupied the
entry number 93994 in the MFT, has been deleted and, the file '512-14-LFN02',
which occupies the entry 93996 in the MFT, has now been renamed '512-14-LFN01'.
Regarding the
time stamps, the renamed file was created on 29-05-2020 at 16:12:55, (in
green). However, as you can see, by renaming it with the same file name as the
deleted one, it acquires the creation date of that deleted file, at 16:12:45,
(in red), showing, in the other timestamps, the original creation date of the
file. This indicates that, in this case, with a waiting time of 14000
milliseconds, a tunnel effect has been generated.
Create a file 'A' in a folder; create a file 'B', in
the same folder; delete file 'A'; and rename file 'B' as 'A
- Create a 512-byte 'A' file with a long name
- Create a 512-byte 'B' file with a long name
- Delete the 'A' file
- Set a timeout of 15000 milliseconds
- Rename file 'B' to 'A
In this test, as
you can see in the previous animated image, a file with the long name
'512-15-LFN01' was created in the system in the 'Test' folder, on 01-06-2020 at
10:23:22. This file was assigned the entry number 93893 in the MFT. After
waiting 10 seconds, a second file was created with the long name
'512-15-LFN02', in the same folder, at 10:23:32. This second file was assigned
entry number 94000 in the MFT.
After waiting
300 seconds, the file was deleted with the name '512-15-LFN01' and, after a
wait of 15000 milliseconds, the file was renamed '512-15-LFN02' to
'512-15-LFN01'.
As you can see
from the above table extracts, the file '512-15-LFN01', which occupied the
entry number 93893 in the MFT, has been deleted and, the file '512-15-LFN02',
which occupies the entry 94000 in the MFT, has now been renamed '512-15-LFN01'.
As far as time
stamps are concerned, in both cases the original dates are shown for each of
the two files, so, with a timeout of 15000 milliseconds, no tunnel effect has
been generated.
Create a file 'A' in a folder; create a file 'B', in
the same folder; delete file 'A'; and rename file 'B' as 'A
- Create a 512-byte 'A' file with a long name
- Create a 512-byte 'B' file with a short name
- Delete the 'A' file
- Set a timeout of 14000 milliseconds
- Rename file 'B' to 'A
In this test, as
you can see in the previous animated image, a file with the long name
'512-14-LFN01' was created in the system in the 'Test' folder, on 30-05-2020 at
15:05:08. This file was assigned the entry number 93893 in the MFT. After
waiting 10 seconds, a second file with the short name '512-14~2', which is the
equivalent of a short name 8.3, was created in the same folder at 15:05:18.
This second file was assigned the entry number 93996 in the MFT.
After waiting
for 300 seconds, the file was deleted with the name '512-14-LFN01' and, after a
wait of 14000 milliseconds, the file '512-14~2' was renamed to '512-14~2',
which is the equivalent of the short name 8.3 of the file '512-14-LFN01'.
As you can see
from the above table extracts, the file '512-14-LFN01', which occupied the
entry number 93893 in the MFT, has been deleted and, the file '512-14~2', which
occupies the entry 93996 in the MFT, has now been renamed '512-14~1'.
Regarding the time
stamps, in both cases the original dates are shown for each of the two files,
so in this case, with a waiting time of 14000 milliseconds, no tunnel effect
has been generated.
Create a file 'A' in a folder; create a file 'B', in
the same folder; delete file 'A'; and rename file 'B' as 'A
- Create a 512-byte 'A' file with a short name
- Create a 512-byte 'B' file with a short name
- Delete the 'A' file
- Set a timeout of 14000 milliseconds
- Rename file 'B' to 'A
In this test, as
you can see in the previous animated image, a file with the short name
'512-14~1' was created in the system in the 'Test' folder, on 01-06-2020 at
05:26:59. This file was assigned the entry number 93893 in the MFT. After
waiting for 10 seconds, a second file with the long name '512-14~2' was created
in the same folder at 05:27:09.
After waiting
300 seconds, the file was deleted with the name '512-14~1' and after a wait of
14000 milliseconds, the file was renamed '512-14~2' to '512-14~1'.
As you can see
from the table extracts above, the file '512-14~1', which originally occupied
the entry number 93893, is no longer available in the MFT, as that entry has
been replaced by another one that now belongs to the file 'ht_settings.ini',
from the Active Disk Editor application, at 05:33:08. On the other hand, the
file '512-14~2', which occupies the 94000 entry in the MFT, has now been
renamed '512-14~1'.
As far as
timestamps are concerned, the renamed file was created on 01-06-2020 at
05:27:09, (in green). However, as you can see, when you rename it with the same
file name as the deleted one, it acquires the creation date of that deleted
file, at 05:26:59, (in red), showing, in the other timestamps, the original
creation date of the file. This indicates that, in this case, with a waiting
time of 14000 milliseconds, a tunnel effect has been generated.
Create a file 'A' in a folder; create a file 'B', in
the same folder; delete file 'A'; and rename file 'B' as 'A
- Create a 512-byte 'A' file with a short name
- Create a 512-byte 'B' file with a short name
- Delete the 'A' file
- Set a timeout of 15000 milliseconds
- Rename file 'B' to 'A
In this test, as
you can see in the previous animated image, a file with the short name
'512-15~1' was created in the system in the 'Test' folder, on 02-06-2020 at
09:09:28. This file was assigned the entry number 93893 in the MFT. After
waiting for 10 seconds, a second file with the short name '512-15~1' was
created in the same folder at 09:09:38. This second file was assigned the entry
number 94000 in the MFT.
After waiting 300 seconds, the file was deleted with the name '512-15~1' and after a wait of 15000 milliseconds, the file was renamed '512-15~2' to '512-15~1'.