codex-admin/autopsy
1
0
Fork 0
mirror of https://github.com/elisspace/autopsy.git synced 2026-09-06 02:24:30 +00:00
Code Issues Releases Wiki Activity

Merge git://github.com/sleuthkit/autopsy

Browse Source
This commit is contained in:
dhurd
2012-09-14 12:24:15 -04:00
parent 24f0851e16 849ee5dd76
commit 0c92f054fc
46 changed files with 952 additions and 506 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
Case/src/org/sleuthkit/autopsy/casemodule/package.dox
View File

@@ -2,20 +2,23 @@
/**
\package org.sleuthkit.autopsy.casemodule
The org.sleuthkit.autopsy.casemodule Module is responsible for organizing a case. A case contains one or more disk images and is the highest-level unit of an investigation.
The org.sleuthkit.autopsy.casemodule Module is responsible for organizing a case. A case contains one or more disk images and is the highest-level unit of an investigation.
\section casemodule_contents Package Contents
The important classes in this case are:
- Case
\section casemodule_overview Overview
All data in a case will be stored in a single database and configuration file. A case must be open before analysis can occur. You will use a Case object to get access to the data being analyzed.
Case settings are stored in an XML file. See the XMLCaseManagement class for more details.
Currently, only one case can be opened at a time. To determine the open case, use the Case.getCurrentCase() method.
Do not cache the case handle object obtained from this method (for example, in member variables) unless you are sure your are acting within case context;
It is safer to call the method more frequently to ensure the validity of the case handle object as new cases are opened.
Once you have the object for the currently open case, Case.getRootObjects() will return the top-level Sleuth Kit Content modules. You can then get their children to go down the tree of data types.

91
CoreComponentInterfaces/src/org/sleuthkit/autopsy/corecomponentinterfaces/package.dox
View File

@@ -1,93 +1,8 @@
/**
* \package org.sleuthkit.autopsy.corecomponentinterfaces
* This package contains the interface classes that define the core components in Autopsy. These components are used in the difference zones of the GUI.
*
<em>NOTE: Much of this data may be out of date. It has not been updated
since a major redesign was done. </em>
\package org.sleuthkit.autopsy.corecomponentinterfaces
\section corecomponentinterface_contents Package Contents
The following are imporant classes in this package
- TODO
This package contains the interface classes that define the core components in Autopsy. These components are used in the different zones of the GUI.
See \ref design_page for examples of the modules and such that it defines.
* <h2>Autopsy Zones</h2>
* There are three major zones in the Autopsy UI. The left hand side has the org.sleuthkit.autopsy.corecomponentinterfaces.DataExplorer zone. This area is where you can search for and explore data. It has all of the analysis smarts. An example of a DataExplorer is the directory tree that shows the hierarchy of directories (and hides the files from view).
* The DataExplorer area identifies a subset of the data to show the user and passes the data to the org.sleuthkit.autopsy.corecomponentinterfaces.DataResult area in the upper right. In the previous example, the contents of a specific folder would be passed to this area and displayed in a table or thumbnail form.</p>
* When a file or object is selected in the DataResult, it is passed to the org.sleuthkit.autopsy.corecomponentinterfaces.DataContent zone in the lower right. This is where file content can be viewed in hex form, strings, etc.
* <h2>Data Flow</h2>
* <h3>Creating Nodes in DataExplorer</h3>
* Data flows between the areas inside of a NetBeans node. The DataExplorer modules create the NetBeans nodes. They query the SQLite database or do whatever they want to identify the set of files that are of interest. They create the NetBeans nodes based on Sleuthkit data model objects. See the org.sleuthkit.autopsy.datamodel package for more details on this.
* <h3>Getting Nodes to DataResult</h3>
* Each DataExplorer TopComponent is responsible for creating its own DataResult TopComponent to display its results. It can choose to re-use the same TopComponent for multiple searches (as DirectoryTree does) or it can choose to make a new one each time (as FileSearch does). The setNode() method on the DataResult object is used to set the root node to display. A dummy root node must be created as the parent if a parent does not already exist.
* The DataExplorer is responsible for setting the double-click and right-click actions associated with the node. The default single click action is to pass data to DataContent. To override this, you must create a new DataResultViewer instance that overrides the propertyChange() method. The DataExplorer adds actions to wrapping the node in a FilterNode variant. The FilterNode then defines the actions for the node by overriding the getPreferredAction() and getActions() methods. As an example, org.sleuthkit.autopsy.directorytree.DataResultFilterNode and org.sleuthkit.autopsy.directorytree.DataResultFilterChildren wraps the nodes that are passed over by the DirectoryTree DataExplorer.
* DataResult can send data back to its DataExplorer by making a custom action that looks up it's instance (DataExplorer.getInstance()).
* <h3>Getting Nodes to DataContent </h3>
* A default DataContent viewer is created when a case is opened. To display the contents of a node, it must be passed to a DataContent instance. The default single-click behavior of the DataResultViewers is to lookup the default DataContent TopComponent and pass the selected node to it. See org.sleuthkit.autopsy.corecomponents.AbstractDataResultViewer.propertyChange(PropertyChangeEvent) for details.
* <h2>Creating new Functionality</h2>
* <h3>Creating a DataExplorer</h3>
* <ol>
* <li>Create a module from within NetBeans. It must be dependent on these modules:
* <ul>
* <li>Case
* <li>CoreComponentInterfaces
* <li>CoreComponents
* <li>DataModel
* <li>DialogsAPI (if pop-ups and such are going to be used)
* <li>Explorer & Property Sheet API
* <li>Lookup
* <li>Nodes API
* <li>Setting API
* <li>UI Utilities API
* <li>Utilities API
* <li>Window System API
* </ul>
* <li> Create a class that implements org.sleuthkit.autopsy.corecomponentinterfaces.DataExplorer. We have been making the TopComponent class be the one that implements DataExplorer. Register this class as a DataExplorer service provider by specifying "@ServiceProvider(service=DataExplorer.class)" in the class or using layer.xml.
* <li>Implement the methods required by the DataExplorer interface.
* <li>Register the class to receive property change events from the org.sleuthkit.autopsy.Case module by using its addPropertyChangeListener() method.
* <li>Access case data using the org.sleuthkit.autopsy.Case module.
* <li>Create Nodes for the data objects using the techniques outlined in the previous section.
* <li>Wrap the nodes in FilterNodes to define actions as outlined in the previous section.
* <li>Send results to DataResults using the techniques outlined in the previous section.
* </ol>
* <h3>Creating a DataResultViewer</h3>
* <p>DataResultTopComponent is the high-level window in the DataResult area. Each instance of this loads up all instances of DataResultViewers that have been registered with the system. Example viewers include the table and thumbnail views. If you want to make your own type of viewer, follow the steps below. Note that the table and thumbnail viewers come with Autopsy by default and can be used by all DataExplorers.
* <ol>
* <li>Create a module from within NetBeans. It must be dependent on these modules:
* <ul>
* <li>Case
* <li>CoreComponentInterfaces
* <li>CoreComponents
* <li>DataModel
* <li>DialogsAPI (if pop-ups and such are going to be used)
* <li>Explorer & Property Sheet API
* <li>Lookup
* <li>Nodes API
* <li>Setting API
* <li>UI Utilities API
* <li>Utilities API
* <li>Window System API
* </ul>
* <li>Make a class that extends org.sleuthkit.autopsy.corecomponents.AbstractDataResultViewer and is registered as a service provider for the org.sleuthkit.autopsy.corecomponentinterfaces.DataResultViewer class by specifying "@ServiceProvider(service = DataResultViewer.class)" or by using layer.xml. This class will extend JPanel. </li>
* <li>See the previous sections on default actions.</li>
* </ol>
* <h3>Creating a DataContentViewer</h3>
* DataContentTopComponent is the high-level window in the DataContent area. Each instance of this loads up all instances of DataContentViewers that have been registered with the system. Example viewers include the strings and hexdump views. If you want to make your own type of viewer, follow the steps below. Note that the strings, hexdump, and image viewers come with Autopsy by default and can be used by all DataExplorers and DataResults. You only need to make a new DataContentViewer if these viewers do not satisfy your needs.
* <ol>
* <li>Create a module from within NetBeans. It must be dependent on these modules:
* <ul>
* <li>Case
* <li>CoreComponentInterfaces
* <li>CoreComponents
* <li>DataModel
* <li>DialogsAPI (if pop-ups and such are going to be used)
* <li>Explorer & Property Sheet API
* <li>Lookup
* <li>Nodes API
* <li>Setting API
* <li>UI Utilities API
* <li>Utilities API
* <li>Window System API
* </ul>
* <li>Make a class that implements org.sleuthkit.autopsy.corecomponentinterfaces.DataContentViewer and is registered as a service provider for DataContentViewer.class by specifying "@ServiceProvider(service = DataContentViewer.class)" or by using layer.xml. This class must extend JPanel. </li>
* </ol>
*/

2
CoreComponents/src/org/sleuthkit/autopsy/corecomponents/Bundle.properties
View File

@@ -50,7 +50,7 @@ DataResultTopComponent.numberMatchLabel.text=0
# Product Information panel
LBL_Description=<div style=\"font-size: 12pt; font-family: Verdana, 'Verdana CE', Arial, 'Arial CE', 'Lucida Grande CE', lucida, 'Helvetica CE', sans-serif;\">\n <b>Product Version:</b> {0} <br><b>Sleuth Kit Version:</b> {7} <br><b>Netbeans RCP Build:</b> {8} <br> <b>Java:</b> {1}; {2}<br> <b>System:</b> {3}; {4}; {5}<br><b>Userdir:</b> {6}</div>
LBL_Description=<div style=\"font-size: 12pt; font-family: Verdana, 'Verdana CE', Arial, 'Arial CE', 'Lucida Grande CE', lucida, 'Helvetica CE', sans-serif;\">\n <b>Product Version:</b> {0} ({9}) <br><b>Sleuth Kit Version:</b> {7} <br><b>Netbeans RCP Build:</b> {8} <br> <b>Java:</b> {1}; {2}<br> <b>System:</b> {3}; {4}; {5}<br><b>Userdir:</b> {6}</div>
Format_OperatingSystem_Value={0} version {1} running on {2}
LBL_Copyright=<div style\="font-size\: 12pt; font-family\: Verdana, 'Verdana CE', Arial, 'Arial CE', 'Lucida Grande CE', lucida, 'Helvetica CE', sans-serif; "> The Autopsy Forensic Browser is a graphical interface to the command line digital investigation analysis tools in The Sleuth Kit. <br> <br>Copyright &copy; 2003-2012. For more information, please visit: <a style\="color\: \#1E2A60;" href\="http\://www.sleuthkit.org">http\://www.sleuthkit.org</a>. </div>
URL_ON_IMG=http://www.sleuthkit.org/

2
CoreComponents/src/org/sleuthkit/autopsy/corecomponents/ProductInformationPanel.java
View File

@@ -54,7 +54,7 @@ class ProductInformationPanel extends JPanel implements HyperlinkListener {
jLabel1.setCursor(Cursor.getPredefinedCursor(Cursor.HAND_CURSOR));
description.setText(org.openide.util.NbBundle.getMessage(ProductInformationPanel.class,
"LBL_Description", new Object[]{getProductVersionValue(), getJavaValue(), getVMValue(),
getOperatingSystemValue(), getEncodingValue(), getSystemLocaleValue(), getUserDirValue(), getSleuthKitVersionValue(), Version.getNetbeansBuild()}));
getOperatingSystemValue(), getEncodingValue(), getSystemLocaleValue(), getUserDirValue(), getSleuthKitVersionValue(), Version.getNetbeansBuild(), Version.getBuildType().toString()}));
description.addHyperlinkListener(this);
copyright.addHyperlinkListener(this);
copyright.setBackground(getBackground());

157
CoreUtils/src/org/sleuthkit/autopsy/coreutils/StringExtract.java
View File

@@ -34,9 +34,9 @@ import org.sleuthkit.autopsy.coreutils.StringExtract.StringExtractUnicodeTable.S
* Currently supports UTF-16 LE, UTF-16 BE and UTF8 Latin, Cyrillic, Chinese,
* Arabic
*
* TODO: - process control characters - testing: check non-printable common
* chars sometimes extracted (font?) - handle tie better (when number of chars
* in result is equal)
* TODO: process control characters
*
* TODO: handle tie better (when number of chars in 2 results is equal)
*/
public class StringExtract {
@@ -50,6 +50,8 @@ public class StringExtract {
* currently enabled scripts
*/
private List<SCRIPT> enabledScripts;
private boolean enableUTF8;
private boolean enableUTF16;
/**
* supported scripts, can be overridden with enableScriptX methods
*/
@@ -73,9 +75,29 @@ public class StringExtract {
throw new IllegalStateException("Unicode table not properly initialized, cannot instantiate StringExtract");
}
this.setEnabledScripts(SUPPORTED_SCRIPTS);
setEnabledScripts(SUPPORTED_SCRIPTS);
enableUTF8 = true;
enableUTF16 = true;
}
public boolean isEnableUTF8() {
return enableUTF8;
}
public void setEnableUTF8(boolean enableUTF8) {
this.enableUTF8 = enableUTF8;
}
public boolean isEnableUTF16() {
return enableUTF16;
}
public void setEnableUTF16(boolean enableUTF16) {
this.enableUTF16 = enableUTF16;
}
/**
* Sets the enabled scripts to ones provided, resets previous setting
*
@@ -85,6 +107,7 @@ public class StringExtract {
this.enabledScripts = scripts;
}
/**
* Sets the enabled script to one provided, resets previous setting
*
@@ -108,8 +131,8 @@ public class StringExtract {
/**
* Check if extraction of the script is enabled by this instance of the
* utility.
* For LATIN_2 (extended LATIN), enable also LATIN_1, even if it's not explicitely enabled.
* utility. For LATIN_2 (extended LATIN), enable also LATIN_1, even if it's
* not explicitely enabled.
*
* @param script script that was identified, to check if it is enabled
* @return true if the the script extraction is enabled
@@ -118,12 +141,25 @@ public class StringExtract {
if (script.equals(SCRIPT.LATIN_1)) {
return enabledScripts.contains(SCRIPT.LATIN_1)
|| enabledScripts.contains(SCRIPT.LATIN_2);
}
else {
} else {
return enabledScripts.contains(script);
}
}
/**
* Determine if Basic Latin/English extraction is set enabled only
* @return true if only Basic Latin/English extraction is set enabled only
*/
public boolean isExtractionLatinBasicOnly() {
if (enabledScripts.size() == 1
&& enabledScripts.get(0).equals(SCRIPT.LATIN_1)) {
return true;
}
else {
return false;
}
}
public static List<SCRIPT> getSupportedScripts() {
return SUPPORTED_SCRIPTS;
@@ -139,6 +175,10 @@ public class StringExtract {
* additional info
*/
public StringExtractResult extract(byte[] buff, int len, int offset) {
if (this.enableUTF16 == false && this.enableUTF8 == false) {
return new StringExtractResult();
}
final int buffLen = buff.length;
int processedBytes = 0;
@@ -162,16 +202,28 @@ public class StringExtract {
//extract using all methods and see which one wins
StringExtractResult resUTF16 = null;
if (curOffset % 2 == 0) {
if (enableUTF16 && curOffset % 2 == 0) {
StringExtractResult resUTF16En1 = extractUTF16(buff, len, curOffset, true);
StringExtractResult resUTF16En2 = extractUTF16(buff, len, curOffset, false);
resUTF16 = resUTF16En1.numChars > resUTF16En2.numChars ? resUTF16En1 : resUTF16En2;
}
}
//results.add(extractUTF8(buff, len, curOffset));
StringExtractResult resUTF8 = extractUTF8(buff, len, curOffset);
StringExtractResult resUTF8 = null;
StringExtractResult resWin;
resWin = resUTF16 != null && resUTF16.numChars > resUTF8.numChars ? resUTF16 : resUTF8;
if (enableUTF8) {
resUTF8 = extractUTF8(buff, len, curOffset);
}
StringExtractResult resWin = null;
if (enableUTF8 && enableUTF16) {
resWin = resUTF16 != null && resUTF16.numChars > resUTF8.numChars ? resUTF16 : resUTF8;
} else if (enableUTF16){
resWin = resUTF16;
}
else if (enableUTF8) {
resWin = resUTF8;
}
if (resWin.numChars >= MIN_CHARS_STRING) {
//record string
@@ -189,9 +241,12 @@ public class StringExtract {
processedBytes += resWin.numBytes;
firstUnprocessedOff = resWin.offset + resWin.numBytes;
} else {
//if no encodings worked, advance 1 byte
++curOffset;
//++processedBytes;
//if no encodings worked, advance byte
if (enableUTF8 == false) {
curOffset += 2;
} else {
++curOffset;
}
}
}
@@ -237,7 +292,7 @@ public class StringExtract {
byteVal += b[0];
//skip if beyond range
if (byteVal > StringExtractUnicodeTable.getUnicodeTableSize() - 1) {
if (byteVal > StringExtractUnicodeTable.UNICODE_TABLE_SIZE - 1) {
break;
}
@@ -421,7 +476,7 @@ public class StringExtract {
curOffset += chBytes;
//skip if beyond range
if (ch > StringExtractUnicodeTable.getUnicodeTableSize() - 1) {
if (ch > StringExtractUnicodeTable.UNICODE_TABLE_SIZE - 1) {
break;
}
@@ -478,6 +533,69 @@ public class StringExtract {
return res;
}
/*
* Extract UTF8/16 ASCII characters from byte buffer - only works for Latin, but fast
*
* The definition of printable are:
* -- All of the letters, numbers, and punctuation.
* -- space and tab
* -- It does NOT include newlines or control chars.
* -- When looking for ASCII strings, they evaluate each byte and when they find four or more printable characters they get printed out with a newline in between each string.
* -- When looking for Unicode strings, they evaluate each two byte sequence and look for four or more printable characters…
*
* @param readBuf the bytes that the string read from
* @param len buffer length
* @param offset offset to start converting from
*
*/
public static String extractASCII(byte[] readBuf, int len, int offset) {
final StringBuilder result = new StringBuilder();
StringBuilder temp = new StringBuilder();
int curLen = 0;
final char NL = (char) 10; // ASCII char for new line
final String NLS = Character.toString(NL);
boolean singleConsecZero = false; //preserve the current sequence of chars if 1 consecutive zero char
for (int i = offset; i < len; i++) {
char curChar = (char) readBuf[i];
if (curChar == 0 && singleConsecZero == false) {
//preserve the current sequence if max consec. 1 zero char
singleConsecZero = true;
} else {
singleConsecZero = false;
}
//ignore non-printable ASCII chars
if (isPrintableAscii(curChar)) {
temp.append(curChar);
++curLen;
} else if (!singleConsecZero) {
if (curLen >= MIN_CHARS_STRING) {
// add to the result and also add the new line at the end
result.append(temp);
result.append(NLS);
}
// reset the temp and curLen
temp = new StringBuilder();
curLen = 0;
}
}
result.append(temp);
return result.toString();
}
/**
* Determine if char is a printable ASCII char
* in range <32,126> and a tab
* @param c char to test
* @return true if it's a printable char, or false otherwise
*/
public static boolean isPrintableAscii(char c) {
return (c >= 32 && c <= 126) || c == 9;
}
/**
* Representation of the string extraction result
@@ -549,10 +667,11 @@ public class StringExtract {
}
},
LATIN_1 {
@Override
@Override
public String toString() {
return "Latin - Basic";
}
@Override
public String getLanguages() {
return "English";

67
DataModel/src/org/sleuthkit/autopsy/datamodel/DataConversion.java
View File

@@ -109,72 +109,7 @@ public class DataConversion {
}
}
/*
* Gets only the printable string from the given characters
*
* The definition of printable are:
* -- All of the letters, numbers, and punctuation.
* -- space and tab
* -- It does NOT include newlines or control chars.
* -- When looking for ASCII strings, they evaluate each byte and when they find four or more printable characters they get printed out with a newline in between each string.
* -- When looking for Unicode strings, they evaluate each two byte sequence and look for four or more printable characters…
*
* @param readBuf the bytes that the string read from
* @param len length of text in the buffer to convert, starting at position 0
* @param minStringLen minimum length of consecutive chars to qualify as a string
*
* TODO should be encoding specific and detect UTF8, UTF16LE, UTF16BE
* then process remainder of the string using detected encoding
*
* @author jantonius
*/
public static String getString(byte[] readBuf, int len, int minStringLen) {
final StringBuilder result = new StringBuilder();
StringBuilder temp = new StringBuilder();
int curLen = 0;
final char NL = (char) 10; // ASCII char for new line
final String NLS = Character.toString(NL);
boolean singleConsecZero = false; //preserve the current sequence of chars if 1 consecutive zero char
for (int i = 0; i < len; i++) {
char curChar = (char) readBuf[i];
if (curChar == 0 && singleConsecZero == false) {
//preserve the current sequence if max consec. 1 zero char
singleConsecZero = true;
} else {
singleConsecZero = false;
}
//ignore non-printable ASCII chars
if (isPrintableAscii(curChar)) {
temp.append(curChar);
++curLen;
} else if (!singleConsecZero) {
if (curLen >= minStringLen) {
// add to the result and also add the new line at the end
result.append(temp);
result.append(NLS);
}
// reset the temp and curLen
temp = new StringBuilder();
curLen = 0;
}
}
result.append(temp);
return result.toString();
}
/**
* Determine if char is a printable ASCII char
* in range <32,126> and a tab
* @param c char to test
* @return true if it's a printable char, or false otherwise
*/
public static boolean isPrintableAscii(char c) {
return (c >= 32 && c <= 126) || c == 9;
}
/**
* Converts the given paths into the formatted path. This mainly used for
* the paths for the "new directory table" and "new output view".

15
DataModel/src/org/sleuthkit/autopsy/datamodel/package.dox
View File

@@ -1,15 +1,16 @@
/**
* \package org.sleuthkit.autopsy.datamodel
\package org.sleuthkit.autopsy.datamodel
Contains the NetBeans Node wrappers that wrap around Sleuth Kit data model objects.
* <h2>Overview</h2>
* Autopsy 3 uses NetBeans Nodes to pass data around. The Sleuth Kit comes with Java datamodel classes in org.sleuthkit.datamodel and it contains classes for files, directories, file systems, volumes, and other data types that can be found in a disk image. These classes are not NetBeans specific.
\section datamodel_overview Overview
Autopsy 3 uses NetBeans Nodes to pass data around. The Sleuth Kit comes with Java datamodel classes in org.sleuthkit.datamodel and it contains classes for files, directories, file systems, volumes, and other data types that can be found in a disk image. These classes are not NetBeans specific.
* This package, org.sleuthkit.autopsy.datamodel, contains classes that are NetBeans-specific and map to classes in the Sleuth Kit datamodel. For example org.sleuthkit.autopsy.datamodel.DirectoryNode is the Node class for the org.sleuthkit.datamodel.Directory class.
This package, org.sleuthkit.autopsy.datamodel, contains classes that are NetBeans-specific and map to classes in the Sleuth Kit datamodel. For example org.sleuthkit.autopsy.datamodel.DirectoryNode is the Node class for the org.sleuthkit.datamodel.Directory class.
* The underlying sleuthkit.datamodel Content objects are passed around Autopsy by being included in a Node's lookup.
The underlying sleuthkit.datamodel Content objects are passed around Autopsy by being included in a Node's lookup.
\section datamodel_creating Creating Nodes
You should only have to create the root node in a hierarchy. To do so, use the RootContentChildren class and pass in the list of TSK datamodel objects that you need to encapsulate. After that, the children will automatically be created as nodes when they are requested. ContentChildren deals with this.
* <h2>Creating Nodes</h2>
* You should only have to create the root node in a hierarchy. To do so, use the RootContentChildren class and pass in the list of TSK datamodel objects that you need to encapsulate. After that, the children will automatically be created as nodes when they are requested. ContentChildren deals with this.
*/

29
DirectoryTree/src/org/sleuthkit/autopsy/directorytree/DirectoryTreeFilterChildren.java
View File

@@ -28,9 +28,12 @@ import org.sleuthkit.autopsy.datamodel.DisplayableItemNode;
import org.sleuthkit.autopsy.datamodel.DisplayableItemNodeVisitor;
import org.sleuthkit.autopsy.datamodel.FileNode;
import org.sleuthkit.autopsy.datamodel.LayoutFileNode;
import org.sleuthkit.autopsy.datamodel.VolumeNode;
import org.sleuthkit.datamodel.Content;
import org.sleuthkit.datamodel.Directory;
import org.sleuthkit.datamodel.LayoutFile;
import org.sleuthkit.datamodel.TskException;
import org.sleuthkit.datamodel.Volume;
/**
* This class wraps around nodes that are displayed in the directory tree and
@@ -82,6 +85,7 @@ class DirectoryTreeFilterChildren extends FilterNode.Children {
/**
* Don't show expansion button on leaves leaf: all children are (file) or
* (directory named "." or "..")
*
* @param node
* @return whether node is a leaf
*/
@@ -104,6 +108,26 @@ class DirectoryTreeFilterChildren extends FilterNode.Children {
return ret;
}
private static boolean isLeafVolume(VolumeNode node) {
Volume vol = node.getLookup().lookup(Volume.class);
boolean ret = true;
try {
for (Content c : vol.getChildren()) {
if (! (c instanceof LayoutFile) ){
ret = false;
break;
}
}
} catch (TskException ex) {
Logger.getLogger(DirectoryTreeFilterChildren.class.getName())
.log(Level.WARNING, "Error getting volume children", ex);
return false;
}
return ret;
}
/**
* Helper to ignore the '.' and '..' directories
*/
@@ -139,6 +163,11 @@ class DirectoryTreeFilterChildren extends FilterNode.Children {
public Boolean visit(DirectoryNode dn) {
return isLeafDirectory(dn);
}
@Override
public Boolean visit(VolumeNode vn) {
return isLeafVolume(vn);
}
}
private static class ShowItemVisitor extends DisplayableItemNodeVisitor.Default<Boolean> {

10
HashDatabase/src/org/sleuthkit/autopsy/hashdatabase/Bundle.properties
View File

@@ -7,7 +7,7 @@ HashDbSimplePanel.knownValLabel.text=-
HashDbSimplePanel.notableValLabel.text=-
HashDbMgmtPanel.addNotableButton.text=Add Known Bad Database
HashDbMgmtPanel.removeNotableButton.text=Remove Selected
HashDbSimplePanel.jLabel1.text=Select known bad databases to use during ingest:
HashDbSimplePanel.jLabel1.text=Enable known bad databases for ingest:
HashDbMgmtPanel.nsrlNameLabel.text=Not Configured
HashDbMgmtPanel.setNSRLButton.text=Change
HashDbMgmtPanel.jLabel1.text=Known Bad Database(s):
@@ -20,8 +20,8 @@ HashDbManagementPanel.hashDbNameLabel.text=No database selected
HashDbManagementPanel.locationLabel.text=Location:
HashDbManagementPanel.hashDbLocationLabel.text=No database selected
HashDbManagementPanel.deleteButton.text=Delete
HashDbManagementPanel.useForIngestCheckbox.text=Use during ingest
HashDbManagementPanel.showInboxMessagesCheckBox.text=Send messages to inbox during ingest
HashDbManagementPanel.useForIngestCheckbox.text=Enable for ingest
HashDbManagementPanel.showInboxMessagesCheckBox.text=Enable sending messages to inbox during ingest
HashDbManagementPanel.indexLabel.text=Index Status:
HashDbManagementPanel.indexButton.text=Index
HashDbManagementPanel.ingestRunningLabel.text=
@@ -37,8 +37,8 @@ HashDbAddDatabaseDialog.browseButton.text=Browse
HashDbAddDatabaseDialog.jLabel1.text=Enter the name of the database:
HashDbAddDatabaseDialog.databaseNameTextField.text=
HashDbAddDatabaseDialog.jLabel2.text=Select the type of database:
HashDbAddDatabaseDialog.useForIngestCheckbox.text=Use during ingest
HashDbAddDatabaseDialog.sendInboxMessagesCheckbox.text=Send messages to inbox during ingest
HashDbAddDatabaseDialog.useForIngestCheckbox.text=Enable for ingest
HashDbAddDatabaseDialog.sendInboxMessagesCheckbox.text=Enable sending messages to inbox during ingest
HashDbManagementPanel.importButton.text=Import
HashDbSearchPanel.hashTable.columnModel.title0=MD5 Hashes
HashDbSearchPanel.hashTable.columnModel.title3=Title 4

5
HashDatabase/src/org/sleuthkit/autopsy/hashdatabase/HashDbAddDatabaseDialog.form
View File

@@ -67,7 +67,7 @@
<Component id="useForIngestCheckbox" alignment="0" min="-2" max="-2" attributes="0"/>
<Component id="sendInboxMessagesCheckbox" alignment="0" min="-2" max="-2" attributes="0"/>
</Group>
<EmptySpace min="0" pref="183" max="32767" attributes="0"/>
<EmptySpace min="0" pref="135" max="32767" attributes="0"/>
</Group>
</Group>
</Group>
@@ -199,6 +199,9 @@
<ResourceString bundle="org/sleuthkit/autopsy/hashdatabase/Bundle.properties" key="HashDbAddDatabaseDialog.useForIngestCheckbox.text" replaceFormat="org.openide.util.NbBundle.getMessage({sourceFileName}.class, &quot;{key}&quot;)"/>
</Property>
</Properties>
<Events>
<EventHandler event="actionPerformed" listener="java.awt.event.ActionListener" parameters="java.awt.event.ActionEvent" handler="useForIngestCheckboxActionPerformed"/>
</Events>
</Component>
<Component class="javax.swing.JCheckBox" name="sendInboxMessagesCheckbox">
<Properties>

11
HashDatabase/src/org/sleuthkit/autopsy/hashdatabase/HashDbAddDatabaseDialog.java
View File

@@ -131,6 +131,11 @@ class HashDbAddDatabaseDialog extends javax.swing.JDialog {
useForIngestCheckbox.setSelected(true);
org.openide.awt.Mnemonics.setLocalizedText(useForIngestCheckbox, org.openide.util.NbBundle.getMessage(HashDbAddDatabaseDialog.class, "HashDbAddDatabaseDialog.useForIngestCheckbox.text")); // NOI18N
useForIngestCheckbox.addActionListener(new java.awt.event.ActionListener() {
public void actionPerformed(java.awt.event.ActionEvent evt) {
useForIngestCheckboxActionPerformed(evt);
}
});
sendInboxMessagesCheckbox.setSelected(true);
org.openide.awt.Mnemonics.setLocalizedText(sendInboxMessagesCheckbox, org.openide.util.NbBundle.getMessage(HashDbAddDatabaseDialog.class, "HashDbAddDatabaseDialog.sendInboxMessagesCheckbox.text")); // NOI18N
@@ -171,7 +176,7 @@ class HashDbAddDatabaseDialog extends javax.swing.JDialog {
.addGroup(layout.createParallelGroup(javax.swing.GroupLayout.Alignment.LEADING)
.addComponent(useForIngestCheckbox)
.addComponent(sendInboxMessagesCheckbox))
.addGap(0, 183, Short.MAX_VALUE))))
.addGap(0, 135, Short.MAX_VALUE))))
);
layout.setVerticalGroup(
layout.createParallelGroup(javax.swing.GroupLayout.Alignment.LEADING)
@@ -293,6 +298,10 @@ class HashDbAddDatabaseDialog extends javax.swing.JDialog {
this.dispose();
}//GEN-LAST:event_okButtonActionPerformed
private void useForIngestCheckboxActionPerformed(java.awt.event.ActionEvent evt) {//GEN-FIRST:event_useForIngestCheckboxActionPerformed
// TODO add your handling code here:
}//GEN-LAST:event_useForIngestCheckboxActionPerformed
// Variables declaration - do not modify//GEN-BEGIN:variables
private javax.swing.JButton browseButton;
private javax.swing.ButtonGroup buttonGroup1;

2
HashDatabase/src/org/sleuthkit/autopsy/hashdatabase/HashDbManagementPanel.form
View File

@@ -152,7 +152,7 @@
</Group>
<Group type="102" alignment="1" attributes="0">
<Component id="showInboxMessagesCheckBox" min="-2" max="-2" attributes="0"/>
<EmptySpace pref="21" max="32767" attributes="0"/>
<EmptySpace pref="16" max="32767" attributes="0"/>
<Component id="deleteButton" min="-2" max="-2" attributes="0"/>
</Group>
</Group>

8
HashDatabase/src/org/sleuthkit/autopsy/hashdatabase/HashDbManagementPanel.java
View File

@@ -303,7 +303,7 @@ public class HashDbManagementPanel extends javax.swing.JPanel {
.addGap(0, 0, Short.MAX_VALUE))
.addGroup(rightPanelLayout.createSequentialGroup()
.addComponent(showInboxMessagesCheckBox)
.addPreferredGap(javax.swing.LayoutStyle.ComponentPlacement.RELATED, 21, Short.MAX_VALUE)
.addPreferredGap(javax.swing.LayoutStyle.ComponentPlacement.RELATED, 16, Short.MAX_VALUE)
.addComponent(deleteButton)))
.addContainerGap())
);
@@ -353,6 +353,11 @@ public class HashDbManagementPanel extends javax.swing.JPanel {
}// </editor-fold>//GEN-END:initComponents
private void deleteButtonActionPerformed(java.awt.event.ActionEvent evt) {//GEN-FIRST:event_deleteButtonActionPerformed
if (JOptionPane.showConfirmDialog(null, "This will remove the hash database entry globally (for all Cases).",
"Deleting a Hash Database Entry",
JOptionPane.YES_NO_OPTION,
JOptionPane.WARNING_MESSAGE) == JOptionPane.YES_OPTION) {
int selected = getSelection();
HashDbXML xmlHandle = HashDbXML.getCurrent();
if (xmlHandle.getNSRLSet() != null) {
@@ -365,6 +370,7 @@ public class HashDbManagementPanel extends javax.swing.JPanel {
HashDbXML.getCurrent().removeKnownBadSetAt(selected);
}
hashSetTableModel.resync();
}
}//GEN-LAST:event_deleteButtonActionPerformed
private void hashSetTableKeyPressed(java.awt.event.KeyEvent evt) {//GEN-FIRST:event_hashSetTableKeyPressed

6
HashDatabase/src/org/sleuthkit/autopsy/hashdatabase/HashDbSimplePanel.java
View File

@@ -193,10 +193,10 @@ public class HashDbSimplePanel extends javax.swing.JPanel {
nsrl = HashDbXML.getCurrent().getNSRLSet();
if (nsrl == null || nsrl.getUseForIngest() == false) {
nsrlDbLabelVal.setText("Not Configured");
nsrlDbLabelVal.setText("Disabled");
}
else {
nsrlDbLabelVal.setText("Configured");
nsrlDbLabelVal.setText("Enabled");
}
reloadCalc();
@@ -231,7 +231,7 @@ public class HashDbSimplePanel extends javax.swing.JPanel {
if (columnIndex == 0) {
return "";
} else {
return "Not Configured";
return "Disabled";
}
} else {
HashDb db = xmlHandle.getKnownBadSets().get(rowIndex);

16
Ingest/src/org/sleuthkit/autopsy/ingest/IngestServices.java
View File

@@ -20,11 +20,13 @@
package org.sleuthkit.autopsy.ingest;
import java.util.logging.Logger;
/**
* Services available to ingest modules via singleton instance,
* e.g. for interacting with the ingest manager
* for sending data events, ingest messages, getting configurations, etc.
* e.g. for logging, interacting with the ingest manager
* sending data events notifications, ingest messages, getting configurations, etc.
*
*/
public class IngestServices {
@@ -48,6 +50,16 @@ public class IngestServices {
return instance;
}
/**
* Get a logger to be used by the module to log messages to log files
* @param module module to get the logger for
* @return logger object
*/
public Logger getLogger(IngestModuleAbstract module) {
return Logger.getLogger(module.getName());
}
/**
* Post ingest message
* @param message ingest message to be posted by ingest module

153
Ingest/src/org/sleuthkit/autopsy/ingest/package.dox
View File

@@ -5,161 +5,12 @@
The package provides the ingest module framework. Ingest modules perform data analysis in a multi-threaded approach.
Refer to \ref mod_ingest_page for details on developing ingest modules.
\section ingestmodule_contents Package Contents
The following are important classes in this package:
- IngestManager
- IngestMessageTopComponent
\section ingestmodule_modules Ingest Module Basics
Ingest modules analyze data from a disk image. They typically focus on a specific type of data analysis. The modules are loaded each time that Autopsy starts. The user can choose to enable each module when they add an image to the case.
There are two types of ingest modules.
- Image-level modules are passed in a reference to an image and perform general analysis on it. These modules may query the database for a small set of files.
- File-level modules are passed in a reference to each file. The Ingest Manager chooses which files to pass and when. These modules are intended to analyze most of the files on the system or that want to examine the file content of all files (i.e. to detect file type based on signature instead of file extension).
Modules post their results to the blackboard (@@@ NEED REFERENCE FOR THIS -- org.sleuthkit.datamodel) and can query the blackboard to get the results of previous modules. For example, the hash database lookup module may want to query for a previously calculated hash value.
The IngestManager class is responsible for launching the ingest modules and passing data to them. Modules can send messages to the ingest inbox (REFERENCE) so that users can see when data has been found.
\section ingestmodule_making Making Ingest Modules
Refer to org.sleuthkit.autopsy.ingest.example for sample source code.
\subsection ingestmodule_making_api Module Interface
The first step is to choose the correct module type. Image-level modules will implement the IngestModuleImage interface and file-level modules will implement the IngestModuleAbstractFile interface.
There is a static getDefault() method that is not part of the interface, that every module (whether an image or a file module) needs to implement to return the registered static instance of the module. Refer to example code in example.ExampleAbstractFileIngestModule.getDefault()
File-level modules need to be singleton (only a single instance at a time). To ensure this, make the constructor private. Ensure the default public file module constructor is overridden with the private one. Image-level modules require a public constructor.
The interfaces have several standard methods that need to be implemented. See the interface methods for details.
- IngestModuleAbstract.init() is invoked every time an ingest session starts. A module should support multiple invocations of init() throughout the application life-cycle.
- IngestModuleAbstract.complete() is invoked when an ingest session completes. The module should perform any resource (files, handles, caches) cleanup in this method and submit final results and post a final ingest inbox message.
- IngestModuleAbstract.stop() is invoked on a module when an ingest session is interrupted by the user or by the system.
The method implementation should be similar to complete() in that the module should perform any cleanup work. If there is pending data to be processed or pending results to be reported by the module then the results should be rejected and ignored if stop() is invoked and the module should terminate as early as possible.
- process() method is invoked to analyze the data. The specific method depends on the module type.
Multiple images can be ingested at the same time. The current behavior is that the files from the second image are added to the list of the files from the first image. The impact of this on module development is that a file-level module could be passed in files from different images in consecutive calls to process(). New instances of image-level modules will be created when the second image is added. Therefore, image-level modules should assume that the process() method will be called only once after init() is called.
Every module should support multiple init() - process() - complete(), and init() - process() - stop() invocations.
The modules should also support multiple init() - complete() and init() - stop() invocations,
which can occur if ingest pipeline is started but no work is enqueued for the particular module.
Module developers are encouraged to use the standard java.util.logging.Logger infrastructure to log errors to the Autopsy log.
\subsection ingestmodule_making_process Process Method
The process method is where the work is done in each type of module. Some notes:
- File-level modules will be called on each file in an order determined by the IngestManager. Each module is free to quickly ignore a file based on name, signature, etc. If a module wants to know the return value from a previously run module, it should use the IngestServices.getAbstractFileModuleResult() method.
- Image-level modules are expected not passed in specific files and are expected to query the database to find the files that they are interested in.
\subsection ingestmodule_making_registration Module Registration
Ingest modules need to be registered using the Netbeans Lookup infrastructure in package's layer.xml file.
An example Image-level module is:
\verbatim
<file name="org-sleuthkit-autopsy-ingest-example-ExampleImageIngestModule.instance">
<attr name="instanceOf" stringvalue="org.sleuthkit.autopsy.ingest.IngestModuleImage"/>
<attr name="instanceCreate" methodvalue="org.sleuthkit.autopsy.ingest.example.ExampleImageIngestModule.getDefault"/>
<attr name="position" intvalue="1000"/>
</file>
\endverbatim
An example file-level module is:
\verbatim
<file name="org-sleuthkit-autopsy-ingest-example-ExampleAbstractFileIngestModule.instance">
<attr name="instanceOf" stringvalue="org.sleuthkit.autopsy.ingest.IngestModuleAbstractFile"/>
<attr name="instanceCreate" methodvalue="org.sleuthkit.autopsy.ingest.example.ExampleAbstractFileIngestModule.getDefault"/>
<attr name="position" intvalue="1100"/>
</file>
\endverbatim
Note the "position" attribute. The attribute determines the ordering of the module in the ingest pipeline.
Modules with lower position attribute will execute earlier.
Use high numbers (higher than 1000) for non-core modules. If your module depends on results from another module, use a higher position attribute to enforce the dependency.
Note: we plan to implement a more flexible and robust module dependency system in future versions of the Autopsy ingest framework.
New modules can be added to the Autopsy ingest pipeline by dropping in jar files into build/cluster/modules.
Dropped in module will be automatically recognized next time Autopsy starts.
\subsection ingestmodule_making_results Posting Results
Users will see the results from ingest modules in one of two ways:
- Results are posted to the blackboard and will be displayed in the navigation tree
- Messages are sent to the Ingest Inbox to notify a user of what has recently been found.
See the Blackboard (REFERENCE) documentation for posting results to it. Modules are free to immediately post results when they find them or they can wait. The IngestServices.getUpdateFrequency() method returns the maximum amount of time that a module can wait before it posts its results.
An example of waiting to post results is the keyword search module. It is resource intensive to commit the keyword index and do a keyword search. Therefore, when its process() method is invoked, it checks if it is close to the getUpdateFrequency() since the last time it did a keyword search. If it is, then it commits the index and performs the search.
When they add data to the blackboard, modules should notify listeners of the new data by periodically invoking IngestServices.fireModuleDataEvent() method. This allows other modules (and the main UI) to know when to query the blackboard for the latest data.
Modules should post messages to the inbox when interesting data is found. The messages includes the module name, message subject, message details, a unique message id (in the context of the originating module), and a uniqueness attribute. The uniqueness attribute is used to group similar messages together and to determine the overall importance priority of the message (if the same message is seen repeatedly, it is considered lower priority).
It is important though to not fill up the inbox with messages. These messages should only be sent if the result has a low false positive rate and will likely be relevant. For example, the hash lookup module will send messages if known bad (notable) files are found, but not if known good (NSRL) files are found. The keyword search module will send messages if a specific keyword matches, but will not send messages (by default) if a regular expression match for a URL has matches (because a lot of the URL hits will be false positives and can generate thousands of messages on a typical system).
Ingest messages have different types: there are info messages, warning messages, error messages and data messages.
The data messages contain encapsulated blackboard artifacts and attributes. The passed in data is used by the ingest inbox GUI widget to navigate to the artifact view in the directory tree, if requested by the user.
Ingest message API is defined in IngestMessage class. The class also contains factory methods to create new messages.
Messages are posted using IngestServices.postMessage() method, which accepts a message object created using one of the factory methods.
Modules should post inbox messages to the user when stop() or complete() is invoked (refer to the examples).
It is recommended to populate the description field of the complete inbox message to provide feedback to the user
summarizing the module ingest run and if any errors were encountered.
\subsection ingestmodule_making_configuration Module Configuration
Ingest modules may require user configuration. The framework
supports two levels of configuration: run-time and general. Run-time configuration
occurs when the user selects which ingest modules to run when an image is added. This level
of configuration should allow the user to enable or disable settings. General configuration is more in-depth and
may require an interface that is more powerful than simple check boxes.
As an example, the keyword search module uses both configuration methods. The run-time configuration allows the user
to choose which lists of keywords to search for. However, if the user wants to edit the lists or create lists, they
need to do go the general configuration window.
Module configuration is decentralized and module-specific; every module maintains its
own configuration state and is responsible for implementing the graphical interface.
The run-time configuration (also called simple configuration), is achieved by each
ingest module providing a JPanel. The IngestModuleAbstract.hasSimpleConfiguration(),
IngestModuleAbstract.getSimpleConfiguration(), and IngestModuleAbstract.saveSimpleConfiguration()
methods should be used for run-time configuration.
The general configuration is also achieved by the module returning a JPanel. A link will be provided to the general configuration from the ingest manager if it exists.
The IngestModuleAbstract.hasAdvancedConfiguration(),
IngestModuleAbstract.getAdvancedConfiguration(), and IngestModuleAbstract.saveAdvancedConfiguration()
methods should be used for general configuration.
\section ingestmodule_events Getting Ingest Status and Events
Other modules and core Autopsy classes may want to get the status of the ingest manager. The IngestManager provides access to this data with the sleuthkit.autopsy.ingest.IngestManager.isIngestRunning() method.
External modules can also register themselves as ingest module event listeners and receive event notifications (when a module is started, stopped, completed or has new data).
Use the IngestManager.addPropertyChangeListener() method to register a module event listener.
Events types received are defined in IngestManager.IngestModuleEvent enum.
At the end of the ingest, IngestManager itself will notify all listeners of IngestModuleEvent.COMPLETED event.
The event is an indication for listeners to perform the final data refresh by quering the blackboard.
Module developers are encouraged to generate periodic IngestModuleEvent.DATA
ModuleDataEvent events when they post data to the blackboard,
but the IngestManager will make a final event to handle scenarios where the module did not notify listeners while it was running.
*/

21
KeywordSearch/src/org/sleuthkit/autopsy/keywordsearch/AbstractFileExtract.java
View File

@@ -20,6 +20,7 @@
package org.sleuthkit.autopsy.keywordsearch;
import java.util.List;
import java.util.Map;
import org.sleuthkit.autopsy.coreutils.StringExtract.StringExtractUnicodeTable.SCRIPT;
import org.sleuthkit.datamodel.AbstractFile;
@@ -28,6 +29,14 @@ import org.sleuthkit.datamodel.AbstractFile;
* chunks
*/
interface AbstractFileExtract {
/**
* Common options that can be used by some extractors
*/
enum ExtractOptions {
EXTRACT_UTF16, ///< extract UTF16 text, possible values Boolean.TRUE.toString(), Boolean.FALSE.toString()
EXTRACT_UTF8, ///< extract UTF8 text, possible values Boolean.TRUE.toString(), Boolean.FALSE.toString()
};
/**
* Get number of chunks resulted from extracting this AbstractFile
@@ -62,6 +71,18 @@ interface AbstractFileExtract {
*/
List<SCRIPT> getScripts();
/**
* Get current options
* @return currently used, extractor specific options, or null of not supported
*/
Map<String,String> getOptions();
/**
* Set extractor specific options
* @param options options to use
*/
void setOptions(Map<String,String> options);
/**
* Determines if the extractor works only for specified types
* is supportedTypes() or whether is a generic content extractor (such as string extractor)

11
KeywordSearch/src/org/sleuthkit/autopsy/keywordsearch/AbstractFileHtmlExtract.java
View File

@@ -23,6 +23,7 @@ import java.io.InputStream;
import java.io.Reader;
import java.nio.charset.Charset;
import java.util.List;
import java.util.Map;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.sleuthkit.autopsy.coreutils.StringExtract.StringExtractUnicodeTable.SCRIPT;
@@ -66,6 +67,16 @@ public class AbstractFileHtmlExtract implements AbstractFileExtract {
public List<SCRIPT> getScripts() {
return null;
}
@Override
public Map<String, String> getOptions() {
return null;
}
@Override
public void setOptions(Map<String, String> options) {
}
@Override
public int getNumChunks() {

41
KeywordSearch/src/org/sleuthkit/autopsy/keywordsearch/AbstractFileStringExtract.java
View File

@@ -22,7 +22,9 @@ import java.io.IOException;
import java.io.InputStream;
import java.nio.charset.Charset;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.sleuthkit.autopsy.coreutils.StringExtract.StringExtractUnicodeTable.SCRIPT;
@@ -52,6 +54,7 @@ class AbstractFileStringExtract implements AbstractFileExtract {
private static final SCRIPT DEFAULT_SCRIPT = SCRIPT.LATIN_2;
private final List<SCRIPT> extractScripts = new ArrayList<SCRIPT>();
private Map<String,String> extractOptions = new HashMap<String,String>();
static {
//prepend UTF-8 BOM to start of the buffer
@@ -89,16 +92,46 @@ class AbstractFileStringExtract implements AbstractFileExtract {
return sourceFile;
}
@Override
public Map<String, String> getOptions() {
return extractOptions;
}
@Override
public void setOptions(Map<String, String> options) {
this.extractOptions = options;
}
@Override
public boolean index(AbstractFile sourceFile) throws IngesterException {
this.sourceFile = sourceFile;
this.numChunks = 0; //unknown until indexing is done
boolean success = false;
//construct stream that extracts text as we read it
//final InputStream stringStream = new AbstractFileStringStream(sourceFile, INDEX_CHARSET);
final InputStream stringStream = new AbstractFileStringIntStream(
sourceFile, extractScripts, INDEX_CHARSET);
final boolean extractUTF8 =
Boolean.parseBoolean(extractOptions.get(AbstractFileExtract.ExtractOptions.EXTRACT_UTF8.toString()));
final boolean extractUTF16 =
Boolean.parseBoolean(extractOptions.get(AbstractFileExtract.ExtractOptions.EXTRACT_UTF16.toString()));
if (extractUTF8 == false && extractUTF16 == false) {
//nothing to do
return true;
}
InputStream stringStream = null;
//check which extract stream to use
if (extractScripts.size() == 1 && extractScripts.get(0).equals(SCRIPT.LATIN_1) ) {
//optimal for english, english only
stringStream = new AbstractFileStringStream(sourceFile, INDEX_CHARSET);
}
else {
stringStream = new AbstractFileStringIntStream(
sourceFile, extractScripts, extractUTF8, extractUTF16, INDEX_CHARSET);
}
try {
success = true;

18
KeywordSearch/src/org/sleuthkit/autopsy/keywordsearch/AbstractFileStringIntStream.java
View File

@@ -48,6 +48,8 @@ public class AbstractFileStringIntStream extends InputStream {
private int convertBuffOffset = 0; //offset to start returning data to user on next read()
private int bytesInConvertBuff = 0; //amount of data currently in the buffer
private boolean fileEOF = false; //if file has more bytes to read
private boolean extractUTF8;
private boolean extractUTF16;
private Charset outCharset;
private static final Logger logger = Logger.getLogger(AbstractFileStringIntStream.class.getName());
private StringExtractResult lastExtractResult;
@@ -59,17 +61,27 @@ public class AbstractFileStringIntStream extends InputStream {
*
* @param content input content to process and turn into a stream to convert into strings
* @param scripts a list of scripts to consider
* @param extractUTF8 whether to extract utf8 encoding
* @param extractUTF16 whether to extract utf16 encoding
* @param outCharset encoding to use in the output byte stream
*/
public AbstractFileStringIntStream(AbstractFile content, List<SCRIPT> scripts, Charset outCharset) {
public AbstractFileStringIntStream(AbstractFile content, List<SCRIPT> scripts, boolean extractUTF8,
boolean extractUTF16, Charset outCharset) {
this.content = content;
this.stringExtractor = new StringExtract();
this.stringExtractor.setEnabledScripts(scripts);
this.extractUTF8 = extractUTF8;
this.extractUTF16 = extractUTF16;
this.outCharset = outCharset;
this.stringExtractor.setEnableUTF8(extractUTF8);
this.stringExtractor.setEnableUTF16(extractUTF16);
}
@Override
public int read() throws IOException {
if (extractUTF8 == false && extractUTF16 == false) {
return -1;
}
final int read = read(oneCharBuf, 0, 1);
if (read == 1) {
return oneCharBuf[0];
@@ -88,6 +100,10 @@ public class AbstractFileStringIntStream extends InputStream {
} else if (len == 0) {
return 0;
}
if (extractUTF8 == false && extractUTF16 == false) {
return -1;
}
long fileSize = content.getSize();
if (fileSize == 0) {

6
KeywordSearch/src/org/sleuthkit/autopsy/keywordsearch/AbstractFileStringStream.java
View File

@@ -20,11 +20,9 @@ package org.sleuthkit.autopsy.keywordsearch;
import java.io.IOException;
import java.io.InputStream;
import java.io.UnsupportedEncodingException;
import java.nio.charset.Charset;
import java.util.logging.Level;
import java.util.logging.Logger;
import org.sleuthkit.autopsy.datamodel.DataConversion;
import org.sleuthkit.autopsy.coreutils.StringExtract;
import org.sleuthkit.datamodel.AbstractFile;
import org.sleuthkit.datamodel.TskException;
@@ -165,7 +163,7 @@ public class AbstractFileStringStream extends InputStream {
} else {
singleConsecZero = false;
}
if (DataConversion.isPrintableAscii(c)) {
if (StringExtract.isPrintableAscii(c)) {
tempString.append(c);
++tempStringLen;
if (tempStringLen >= MIN_PRINTABLE_CHARS) {

11
KeywordSearch/src/org/sleuthkit/autopsy/keywordsearch/AbstractFileTikaTextExtract.java
View File

@@ -25,6 +25,7 @@ import java.nio.charset.Charset;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
@@ -87,6 +88,16 @@ public class AbstractFileTikaTextExtract implements AbstractFileExtract {
@Override
public List<StringExtract.StringExtractUnicodeTable.SCRIPT> getScripts() {
return null;
}
@Override
public Map<String, String> getOptions() {
return null;
}
@Override
public void setOptions(Map<String, String> options) {
}
@Override

17
KeywordSearch/src/org/sleuthkit/autopsy/keywordsearch/Bundle.properties
View File

@@ -24,14 +24,14 @@ KeywordSearchEditListPanel.copyMenuItem.text=Copy
KeywordSearchEditListPanel.exportButton.text=Export List
KeywordSearchEditListPanel.deleteListButton.text=Delete List
KeywordSearchListsManagementPanel.newListButton.text=New List
KeywordSearchEditListPanel.useForIngestCheckbox.text=Use during ingest
KeywordSearchEditListPanel.useForIngestCheckbox.text=Enable for ingest
KeywordSearchListsManagementPanel.importButton.text=Import List
KeywordSearchPanel.searchBox.text=Search...
KeywordSearchPanel.regExCheckboxMenuItem.text=Use Regular Expressions
KeywordSearchPanel.settingsLabel.text=
KeywordSearchListsViewerPanel.searchAddButton.text=Search
KeywordSearchListsViewerPanel.manageListsButton.text=Manage Lists
KeywordSearchIngestSimplePanel.jLabel1.text=Select lists to use during ingest:
KeywordSearchIngestSimplePanel.jLabel1.text=Select keyword lists to enable during ingest:
KeywordSearchListsViewerPanel.ingestIndexLabel.text=Files Indexed:
KeywordSearchEditListPanel.selectorsCombo.toolTipText=Regular Expression selector type (optional)
KeywordSearchPanel.searchButton.text=
@@ -48,16 +48,19 @@ ExtractedContentPanel.pageOfLabel.text=of
ExtractedContentPanel.pageCurLabel.text=-
ExtractedContentPanel.pageTotalLabel.text=-
ExtractedContentPanel.hitLabel.toolTipText=
KeywordSearchEditListPanel.ingestMessagesCheckbox.text=Send messages to inbox during ingest
KeywordSearchEditListPanel.ingestMessagesCheckbox.text=Enable sending messages to inbox during ingest
KeywordSearchEditListPanel.ingestMessagesCheckbox.toolTipText=Send messages during triage / ingest when hits on keyword from this list occur
KeywordSearchConfigurationPanel2.skipNSRLCheckBox.text=Do not add files in NSRL (known files) to keyword index
KeywordSearchConfigurationPanel2.skipNSRLCheckBox.text=Do not add files in NSRL (known files) to keyword index during ingest
KeywordSearchConfigurationPanel2.skipNSRLCheckBox.toolTipText=Requires Hash DB service to had run previously, or be selected for next ingest.
KeywordSearchConfigurationPanel2.filesIndexedValue.text=-
KeywordSearchConfigurationPanel2.filesIndexedLabel.text=Files in keyword index:
KeywordSearchIngestSimplePanel.languagesLabel.text=Scripts to extract from unknown file types (in Advanced settings):
KeywordSearchIngestSimplePanel.languagesLabel.text=Scripts enabled for string extraction from unknown file types:
KeywordSearchIngestSimplePanel.languagesValLabel.text=-
KeywordSearchIngestSimplePanel.languagesLabel.toolTipText=
KeywordSearchIngestSimplePanel.languagesLabel.toolTipText=Scripts enabled for string extraction from unknown file types. Changes can be done in Advanced Settings.
KeywordSearchIngestSimplePanel.languagesValLabel.toolTipText=
KeywordSearchConfigurationPanel3.languagesLabel.text=Scripts to extract from unknown file types (change will be effective on next ingest):
KeywordSearchConfigurationPanel3.languagesLabel.text=Enabled scripts (languages):
KeywordSearchConfigurationPanel2.chunksLabel.text=Chunks in keyword index:
KeywordSearchConfigurationPanel2.chunksValLabel.text=-
KeywordSearchConfigurationPanel3.enableUTF8Checkbox.text=Enable UTF8 text extraction
KeywordSearchConfigurationPanel3.enableUTF16Checkbox.text=Enable UTF16LE and UTF16BE string extraction
KeywordSearchConfigurationPanel3.jLabel1.text=Ingest settings for string extraction from unknown file types (changes effective on next ingest)

2
KeywordSearch/src/org/sleuthkit/autopsy/keywordsearch/KeywordSearchConfigurationPanel.java
View File

@@ -41,7 +41,7 @@ public class KeywordSearchConfigurationPanel extends javax.swing.JPanel {
private void customizeComponents() {
setName("Advanced Keyword Search Configuration");
tabbedPane.insertTab("Lists", null, KeywordSearchConfigurationPanel1.getDefault(), "List configuration", 0);
tabbedPane.insertTab("Languages", null, KeywordSearchConfigurationPanel3.getDefault(), "Languages configuration", 1);
tabbedPane.insertTab("String Extraction", null, KeywordSearchConfigurationPanel3.getDefault(), "String extraction configuration for Keyword Search Ingest", 1);
tabbedPane.insertTab("General", null, KeywordSearchConfigurationPanel2.getDefault(), "General configuration", 2);
}

2
KeywordSearch/src/org/sleuthkit/autopsy/keywordsearch/KeywordSearchConfigurationPanel2.form
View File

@@ -37,7 +37,7 @@
</Group>
</Group>
</Group>
<EmptySpace pref="147" max="32767" attributes="0"/>
<EmptySpace pref="81" max="32767" attributes="0"/>
</Group>
</Group>
</DimensionLayout>

2
KeywordSearch/src/org/sleuthkit/autopsy/keywordsearch/KeywordSearchConfigurationPanel2.java
View File

@@ -117,7 +117,7 @@ public class KeywordSearchConfigurationPanel2 extends javax.swing.JPanel {
.addGroup(layout.createParallelGroup(javax.swing.GroupLayout.Alignment.LEADING, false)
.addComponent(filesIndexedValue, javax.swing.GroupLayout.DEFAULT_SIZE, 104, Short.MAX_VALUE)
.addComponent(chunksValLabel, javax.swing.GroupLayout.DEFAULT_SIZE, javax.swing.GroupLayout.DEFAULT_SIZE, Short.MAX_VALUE))))
.addContainerGap(147, Short.MAX_VALUE))
.addContainerGap(81, Short.MAX_VALUE))
);
layout.setVerticalGroup(
layout.createParallelGroup(javax.swing.GroupLayout.Alignment.LEADING)

67
KeywordSearch/src/org/sleuthkit/autopsy/keywordsearch/KeywordSearchConfigurationPanel3.form
View File

@@ -16,25 +16,37 @@
<Layout>
<DimensionLayout dim="0">
<Group type="103" groupAlignment="0" attributes="0">
<Group type="102" alignment="0" attributes="0">
<EmptySpace max="-2" attributes="0"/>
<Group type="102" attributes="0">
<Group type="103" groupAlignment="0" attributes="0">
<Group type="102" attributes="0">
<Component id="languagesLabel" min="-2" max="-2" attributes="0"/>
<EmptySpace max="32767" attributes="0"/>
<Group type="102" alignment="0" attributes="0">
<EmptySpace max="-2" attributes="0"/>
<Group type="103" groupAlignment="0" max="-2" attributes="0">
<Component id="langPanel" max="32767" attributes="0"/>
<Component id="enableUTF16Checkbox" min="-2" max="-2" attributes="0"/>
<Component id="languagesLabel" alignment="0" max="32767" attributes="0"/>
<Component id="enableUTF8Checkbox" alignment="0" min="-2" max="-2" attributes="0"/>
</Group>
</Group>
<Component id="langPanel" max="32767" attributes="0"/>
<Component id="jLabel1" alignment="0" min="-2" max="-2" attributes="0"/>
</Group>
<EmptySpace min="0" pref="0" max="32767" attributes="0"/>
</Group>
</Group>
</DimensionLayout>
<DimensionLayout dim="1">
<Group type="103" groupAlignment="0" attributes="0">
<Group type="102" alignment="0" attributes="0">
<EmptySpace min="-2" pref="4" max="-2" attributes="0"/>
<Component id="jLabel1" min="-2" max="-2" attributes="0"/>
<EmptySpace type="unrelated" max="-2" attributes="0"/>
<Component id="enableUTF16Checkbox" min="-2" max="-2" attributes="0"/>
<EmptySpace max="-2" attributes="0"/>
<Component id="enableUTF8Checkbox" min="-2" max="-2" attributes="0"/>
<EmptySpace min="-2" pref="4" max="-2" attributes="0"/>
<Component id="languagesLabel" min="-2" max="-2" attributes="0"/>
<EmptySpace max="-2" attributes="0"/>
<Component id="langPanel" max="32767" attributes="0"/>
<Component id="langPanel" min="-2" pref="346" max="-2" attributes="0"/>
<EmptySpace max="-2" attributes="0"/>
</Group>
</Group>
</DimensionLayout>
@@ -48,25 +60,62 @@
</Properties>
</Component>
<Container class="javax.swing.JScrollPane" name="langPanel">
<Properties>
<Property name="preferredSize" type="java.awt.Dimension" editor="org.netbeans.beaninfo.editors.DimensionEditor">
<Dimension value="[430, 361]"/>
</Property>
</Properties>
<Layout class="org.netbeans.modules.form.compat2.layouts.support.JScrollPaneSupportLayout"/>
<SubComponents>
<Container class="javax.swing.JPanel" name="checkPanel">
<Properties>
<Property name="preferredSize" type="java.awt.Dimension" editor="org.netbeans.beaninfo.editors.DimensionEditor">
<Dimension value="[400, 361]"/>
</Property>
</Properties>
<Layout>
<DimensionLayout dim="0">
<Group type="103" groupAlignment="0" attributes="0">
<EmptySpace min="0" pref="434" max="32767" attributes="0"/>
<EmptySpace min="0" pref="411" max="32767" attributes="0"/>
</Group>
</DimensionLayout>
<DimensionLayout dim="1">
<Group type="103" groupAlignment="0" attributes="0">
<EmptySpace min="0" pref="211" max="32767" attributes="0"/>
<EmptySpace min="0" pref="361" max="32767" attributes="0"/>
</Group>
</DimensionLayout>
</Layout>
</Container>
</SubComponents>
</Container>
<Component class="javax.swing.JCheckBox" name="enableUTF8Checkbox">
<Properties>
<Property name="text" type="java.lang.String" editor="org.netbeans.modules.i18n.form.FormI18nStringEditor">
<ResourceString bundle="org/sleuthkit/autopsy/keywordsearch/Bundle.properties" key="KeywordSearchConfigurationPanel3.enableUTF8Checkbox.text" replaceFormat="org.openide.util.NbBundle.getMessage({sourceFileName}.class, &quot;{key}&quot;)"/>
</Property>
</Properties>
<Events>
<EventHandler event="actionPerformed" listener="java.awt.event.ActionListener" parameters="java.awt.event.ActionEvent" handler="enableUTF8CheckboxActionPerformed"/>
</Events>
</Component>
<Component class="javax.swing.JCheckBox" name="enableUTF16Checkbox">
<Properties>
<Property name="text" type="java.lang.String" editor="org.netbeans.modules.i18n.form.FormI18nStringEditor">
<ResourceString bundle="org/sleuthkit/autopsy/keywordsearch/Bundle.properties" key="KeywordSearchConfigurationPanel3.enableUTF16Checkbox.text" replaceFormat="org.openide.util.NbBundle.getMessage({sourceFileName}.class, &quot;{key}&quot;)"/>
</Property>
</Properties>
<Events>
<EventHandler event="actionPerformed" listener="java.awt.event.ActionListener" parameters="java.awt.event.ActionEvent" handler="enableUTF16CheckboxActionPerformed"/>
</Events>
</Component>
<Component class="javax.swing.JLabel" name="jLabel1">
<Properties>
<Property name="text" type="java.lang.String" editor="org.netbeans.modules.i18n.form.FormI18nStringEditor">
<ResourceString bundle="org/sleuthkit/autopsy/keywordsearch/Bundle.properties" key="KeywordSearchConfigurationPanel3.jLabel1.text" replaceFormat="org.openide.util.NbBundle.getMessage({sourceFileName}.class, &quot;{key}&quot;)"/>
</Property>
</Properties>
</Component>
</SubComponents>
</Form>

109
KeywordSearch/src/org/sleuthkit/autopsy/keywordsearch/KeywordSearchConfigurationPanel3.java
View File

@@ -105,22 +105,50 @@ public class KeywordSearchConfigurationPanel3 extends javax.swing.JPanel {
private void reloadScriptsCheckBoxes() {
final KeywordSearchIngestModule service = KeywordSearchIngestModule.getDefault();
boolean utf16 =
Boolean.parseBoolean(service.getStringExtractOption(AbstractFileExtract.ExtractOptions.EXTRACT_UTF16.toString()));
enableUTF16Checkbox.setSelected(utf16);
boolean utf8 =
Boolean.parseBoolean(service.getStringExtractOption(AbstractFileExtract.ExtractOptions.EXTRACT_UTF8.toString()));
enableUTF8Checkbox.setSelected(utf8);
final List<SCRIPT> serviceScripts = service.getStringExtractScripts();
final int components = checkPanel.getComponentCount();
for (int i = 0; i < components; ++i) {
JCheckBox ch = (JCheckBox) checkPanel.getComponent(i);
StringExtract.StringExtractUnicodeTable.SCRIPT script = scripts.get(ch.getText());
ch.setSelected(serviceScripts.contains(script));
}
}
private void activateWidgets() {
reloadScriptsCheckBoxes();
boolean enable = !IngestManager.getDefault().isIngestRunning()
final KeywordSearchIngestModule service = KeywordSearchIngestModule.getDefault();
boolean utf16 =
Boolean.parseBoolean(service.getStringExtractOption(AbstractFileExtract.ExtractOptions.EXTRACT_UTF16.toString()));
enableUTF16Checkbox.setSelected(utf16);
boolean utf8 =
Boolean.parseBoolean(service.getStringExtractOption(AbstractFileExtract.ExtractOptions.EXTRACT_UTF8.toString()));
enableUTF8Checkbox.setSelected(utf8);
final boolean extractEnabled = utf16 || utf8;
boolean ingestNotRunning = !IngestManager.getDefault().isIngestRunning()
&& ! IngestManager.getDefault().isModuleRunning(KeywordSearchIngestModule.getDefault());;
//enable / disable checboxes
activateScriptsCheckboxes(enable);
activateScriptsCheckboxes(extractEnabled && ingestNotRunning);
enableUTF16Checkbox.setEnabled(ingestNotRunning);
enableUTF8Checkbox.setEnabled(ingestNotRunning);
}
/**
@@ -135,45 +163,106 @@ public class KeywordSearchConfigurationPanel3 extends javax.swing.JPanel {
languagesLabel = new javax.swing.JLabel();
langPanel = new javax.swing.JScrollPane();
checkPanel = new javax.swing.JPanel();
enableUTF8Checkbox = new javax.swing.JCheckBox();
enableUTF16Checkbox = new javax.swing.JCheckBox();
jLabel1 = new javax.swing.JLabel();
org.openide.awt.Mnemonics.setLocalizedText(languagesLabel, org.openide.util.NbBundle.getMessage(KeywordSearchConfigurationPanel3.class, "KeywordSearchConfigurationPanel3.languagesLabel.text")); // NOI18N
langPanel.setPreferredSize(new java.awt.Dimension(430, 361));
checkPanel.setPreferredSize(new java.awt.Dimension(400, 361));
javax.swing.GroupLayout checkPanelLayout = new javax.swing.GroupLayout(checkPanel);
checkPanel.setLayout(checkPanelLayout);
checkPanelLayout.setHorizontalGroup(
checkPanelLayout.createParallelGroup(javax.swing.GroupLayout.Alignment.LEADING)
.addGap(0, 434, Short.MAX_VALUE)
.addGap(0, 411, Short.MAX_VALUE)
);
checkPanelLayout.setVerticalGroup(
checkPanelLayout.createParallelGroup(javax.swing.GroupLayout.Alignment.LEADING)
.addGap(0, 211, Short.MAX_VALUE)
.addGap(0, 361, Short.MAX_VALUE)
);
langPanel.setViewportView(checkPanel);
org.openide.awt.Mnemonics.setLocalizedText(enableUTF8Checkbox, org.openide.util.NbBundle.getMessage(KeywordSearchConfigurationPanel3.class, "KeywordSearchConfigurationPanel3.enableUTF8Checkbox.text")); // NOI18N
enableUTF8Checkbox.addActionListener(new java.awt.event.ActionListener() {
public void actionPerformed(java.awt.event.ActionEvent evt) {
enableUTF8CheckboxActionPerformed(evt);
}
});
org.openide.awt.Mnemonics.setLocalizedText(enableUTF16Checkbox, org.openide.util.NbBundle.getMessage(KeywordSearchConfigurationPanel3.class, "KeywordSearchConfigurationPanel3.enableUTF16Checkbox.text")); // NOI18N
enableUTF16Checkbox.addActionListener(new java.awt.event.ActionListener() {
public void actionPerformed(java.awt.event.ActionEvent evt) {
enableUTF16CheckboxActionPerformed(evt);
}
});
org.openide.awt.Mnemonics.setLocalizedText(jLabel1, org.openide.util.NbBundle.getMessage(KeywordSearchConfigurationPanel3.class, "KeywordSearchConfigurationPanel3.jLabel1.text")); // NOI18N
javax.swing.GroupLayout layout = new javax.swing.GroupLayout(this);
this.setLayout(layout);
layout.setHorizontalGroup(
layout.createParallelGroup(javax.swing.GroupLayout.Alignment.LEADING)
.addGroup(layout.createSequentialGroup()
.addContainerGap()
.addGroup(layout.createParallelGroup(javax.swing.GroupLayout.Alignment.LEADING)
.addGroup(layout.createSequentialGroup()
.addComponent(languagesLabel)
.addContainerGap())
.addComponent(langPanel)))
.addContainerGap()
.addGroup(layout.createParallelGroup(javax.swing.GroupLayout.Alignment.LEADING, false)
.addComponent(langPanel, javax.swing.GroupLayout.DEFAULT_SIZE, javax.swing.GroupLayout.DEFAULT_SIZE, Short.MAX_VALUE)
.addComponent(enableUTF16Checkbox)
.addComponent(languagesLabel, javax.swing.GroupLayout.DEFAULT_SIZE, javax.swing.GroupLayout.DEFAULT_SIZE, Short.MAX_VALUE)
.addComponent(enableUTF8Checkbox)))
.addComponent(jLabel1))
.addGap(0, 0, Short.MAX_VALUE))
);
layout.setVerticalGroup(
layout.createParallelGroup(javax.swing.GroupLayout.Alignment.LEADING)
.addGroup(layout.createSequentialGroup()
.addContainerGap()
.addGap(4, 4, 4)
.addComponent(jLabel1)
.addPreferredGap(javax.swing.LayoutStyle.ComponentPlacement.UNRELATED)
.addComponent(enableUTF16Checkbox)
.addPreferredGap(javax.swing.LayoutStyle.ComponentPlacement.RELATED)
.addComponent(enableUTF8Checkbox)
.addGap(4, 4, 4)
.addComponent(languagesLabel)
.addPreferredGap(javax.swing.LayoutStyle.ComponentPlacement.RELATED)
.addComponent(langPanel))
.addComponent(langPanel, javax.swing.GroupLayout.PREFERRED_SIZE, 346, javax.swing.GroupLayout.PREFERRED_SIZE)
.addContainerGap())
);
}// </editor-fold>//GEN-END:initComponents
private void enableUTF8CheckboxActionPerformed(java.awt.event.ActionEvent evt) {//GEN-FIRST:event_enableUTF8CheckboxActionPerformed
final KeywordSearchIngestModule service = KeywordSearchIngestModule.getDefault();
boolean selected = this.enableUTF8Checkbox.isSelected();
service.setStringExtractOption(AbstractFileExtract.ExtractOptions.EXTRACT_UTF8.toString(),
Boolean.toString(selected));
activateScriptsCheckboxes(selected || this.enableUTF16Checkbox.isSelected());
}//GEN-LAST:event_enableUTF8CheckboxActionPerformed
private void enableUTF16CheckboxActionPerformed(java.awt.event.ActionEvent evt) {//GEN-FIRST:event_enableUTF16CheckboxActionPerformed
final KeywordSearchIngestModule service = KeywordSearchIngestModule.getDefault();
boolean selected = this.enableUTF16Checkbox.isSelected();
service.setStringExtractOption(AbstractFileExtract.ExtractOptions.EXTRACT_UTF16.toString(),
Boolean.toString(selected));
activateScriptsCheckboxes(selected || this.enableUTF8Checkbox.isSelected());
}//GEN-LAST:event_enableUTF16CheckboxActionPerformed
// Variables declaration - do not modify//GEN-BEGIN:variables
private javax.swing.JPanel checkPanel;
private javax.swing.JCheckBox enableUTF16Checkbox;
private javax.swing.JCheckBox enableUTF8Checkbox;
private javax.swing.JLabel jLabel1;
private javax.swing.JScrollPane langPanel;
private javax.swing.JLabel languagesLabel;
// End of variables declaration//GEN-END:variables

15
KeywordSearch/src/org/sleuthkit/autopsy/keywordsearch/KeywordSearchEditListPanel.java
View File

@@ -587,9 +587,15 @@ class KeywordSearchEditListPanel extends javax.swing.JPanel implements ListSelec
}//GEN-LAST:event_saveListButtonActionPerformed
private void deleteWordButtonActionPerformed(java.awt.event.ActionEvent evt) {//GEN-FIRST:event_deleteWordButtonActionPerformed
if (KeywordSearchUtil.displayConfirmDialog("Removing a keyword"
, "This will remove a keyword from the list globally (for all Cases). "
+ "Do you want to proceed? "
, KeywordSearchUtil.DIALOG_MESSAGE_TYPE.WARN) ) {
tableModel.deleteSelected(keywordTable.getSelectedRows());
KeywordSearchListsXML.getCurrent().addList(currentKeywordList);
initButtons();
}
}//GEN-LAST:event_deleteWordButtonActionPerformed
private void addWordFieldActionPerformed(java.awt.event.ActionEvent evt) {//GEN-FIRST:event_addWordFieldActionPerformed
@@ -638,17 +644,24 @@ class KeywordSearchEditListPanel extends javax.swing.JPanel implements ListSelec
final KeywordSearchListsXML exporter = new KeywordSearchListsXML(fileAbs);
boolean written = exporter.writeLists(toWrite);
if (written) {
KeywordSearchUtil.displayDialog(FEATURE_NAME, "Keyword lists exported", KeywordSearchUtil.DIALOG_MESSAGE_TYPE.INFO);
KeywordSearchUtil.displayDialog(FEATURE_NAME, "Keyword lists exported",
KeywordSearchUtil.DIALOG_MESSAGE_TYPE.INFO);
}
}
}//GEN-LAST:event_exportButtonActionPerformed
private void deleteListButtonActionPerformed(java.awt.event.ActionEvent evt) {//GEN-FIRST:event_deleteListButtonActionPerformed
if (KeywordSearchUtil.displayConfirmDialog("Delete a keyword list"
, "This will delete the keyword list globally (for all Cases). "
+ "Do you want to proceed with the deletion? "
, KeywordSearchUtil.DIALOG_MESSAGE_TYPE.WARN) ) {
KeywordSearchListsXML deleter = KeywordSearchListsXML.getCurrent();
String toDelete = currentKeywordList.getName();
currentKeywordList = null;
initButtons();
deleter.deleteList(toDelete);
}
}//GEN-LAST:event_deleteListButtonActionPerformed
private void chRegexActionPerformed(java.awt.event.ActionEvent evt) {//GEN-FIRST:event_chRegexActionPerformed

33
KeywordSearch/src/org/sleuthkit/autopsy/keywordsearch/KeywordSearchIngestModule.java
View File

@@ -103,6 +103,8 @@ public final class KeywordSearchIngestModule implements IngestModuleAbstractFile
private List<AbstractFileExtract> textExtractors;
private AbstractFileStringExtract stringExtractor;
private final List<SCRIPT> stringExtractScripts = new ArrayList<SCRIPT>();
private Map<String,String> stringExtractOptions = new HashMap<String,String>();
private final GetIsFileKnownV getIsFileKnown = new GetIsFileKnownV();
private enum IngestStatus {
@@ -115,7 +117,9 @@ public final class KeywordSearchIngestModule implements IngestModuleAbstractFile
private KeywordSearchIngestModule() {
//set default script
stringExtractScripts.add(SCRIPT.LATIN_1);
stringExtractScripts.add(SCRIPT.LATIN_2);
stringExtractOptions.put(AbstractFileExtract.ExtractOptions.EXTRACT_UTF8.toString(), Boolean.TRUE.toString());
stringExtractOptions.put(AbstractFileExtract.ExtractOptions.EXTRACT_UTF16.toString(), Boolean.TRUE.toString());
}
/**
@@ -308,6 +312,9 @@ public final class KeywordSearchIngestModule implements IngestModuleAbstractFile
//initialize extractors
stringExtractor = new AbstractFileStringExtract();
stringExtractor.setScripts(stringExtractScripts);
stringExtractor.setOptions(stringExtractOptions);
//log the scripts used for debugging
final StringBuilder sbScripts = new StringBuilder();
for (SCRIPT s : stringExtractScripts) {
@@ -1073,4 +1080,28 @@ public final class KeywordSearchIngestModule implements IngestModuleAbstractFile
List<SCRIPT> getStringExtractScripts() {
return new ArrayList<SCRIPT>(this.stringExtractScripts);
}
/**
* Set / override string extract option
* @param key option name to set
* @param val option value to set
*/
void setStringExtractOption(String key, String val) {
this.stringExtractOptions.put(key, val);
}
/**
* get string extract option for the key
* @param key option name
* @return option string value, or empty string if the option is not set
*/
String getStringExtractOption(String key) {
if (this.stringExtractOptions.containsKey(key)) {
return this.stringExtractOptions.get(key);
}
else {
return "";
}
}
}

2
KeywordSearch/src/org/sleuthkit/autopsy/keywordsearch/KeywordSearchIngestSimplePanel.form
View File

@@ -33,7 +33,7 @@
</Group>
<Group type="102" attributes="0">
<Component id="languagesLabel" min="-2" max="-2" attributes="0"/>
<EmptySpace min="0" pref="5" max="32767" attributes="0"/>
<EmptySpace min="0" pref="0" max="32767" attributes="0"/>
</Group>
</Group>
<EmptySpace max="-2" attributes="0"/>

2
KeywordSearch/src/org/sleuthkit/autopsy/keywordsearch/KeywordSearchIngestSimplePanel.java
View File

@@ -128,7 +128,7 @@ public class KeywordSearchIngestSimplePanel extends javax.swing.JPanel {
.addComponent(languagesValLabel, javax.swing.GroupLayout.DEFAULT_SIZE, javax.swing.GroupLayout.DEFAULT_SIZE, Short.MAX_VALUE))
.addGroup(layout.createSequentialGroup()
.addComponent(languagesLabel)
.addGap(0, 5, Short.MAX_VALUE)))
.addGap(0, 0, Short.MAX_VALUE)))
.addContainerGap())
.addGroup(layout.createSequentialGroup()
.addGroup(layout.createParallelGroup(javax.swing.GroupLayout.Alignment.LEADING)

26
KeywordSearch/src/org/sleuthkit/autopsy/keywordsearch/Server.java
View File

@@ -211,14 +211,16 @@ class Server {
public void run() {
InputStreamReader isr = new InputStreamReader(stream);
BufferedReader br = new BufferedReader(isr);
final Version.Type builtType = Version.getBuildType();
try {
OutputStreamWriter osw = new OutputStreamWriter(out, PlatformUtil.getDefaultPlatformCharset());
BufferedWriter bw = new BufferedWriter(osw);
String line = null;
while (doRun && (line = br.readLine()) != null) {
bw.write(line);
bw.newLine();
if (Version.getBuildType() == Version.Type.DEVELOPMENT) {
if (builtType == Version.Type.DEVELOPMENT) {
bw.write(line);
bw.newLine();
//if (builtType == Version.Type.DEVELOPMENT) {
//flush buffers if dev version for debugging
bw.flush();
}
@@ -397,11 +399,11 @@ class Server {
return currentCore.queryNumIndexedFiles();
}
/**
* Execute query that gets only number of all Solr file chunks (not logical files) indexed without
* actually returning the content.
*
/**
* Execute query that gets only number of all Solr file chunks (not logical
* files) indexed without actually returning the content.
*
* @return int representing number of indexed chunks
* @throws SolrServerException
*/
@@ -675,10 +677,10 @@ class Server {
private int queryNumIndexedFiles() throws SolrServerException {
return queryNumIndexedDocuments() - queryNumIndexedChunks();
}
/**
* Execute query that gets only number of all chunks (not logical files, or all documents)
* indexed without actually returning the content
/**
* Execute query that gets only number of all chunks (not logical files,
* or all documents) indexed without actually returning the content
*
* @return int representing number of indexed chunks
* @throws SolrServerException

2
NEWS.txt
View File

@@ -1,6 +1,4 @@
3.0.0b5 (September 12, 2012)
Funded by US Army Intelligence Center of Excellence (USAICoE):
New features:
- Added international string extraction from unknown file types.
- Removed size limitations of large files for keyword searching.

12
Report/src/org/sleuthkit/autopsy/report/package.dox
View File

@@ -1,19 +1,9 @@
/**
\package org.sleuthkit.autopsy.report
This package provides the reporting framework. Reporting modules allow you to get output from Autopsy in the form of XML, HTML, etc.
\section report_contents Package Contents
The following are important classes in this package:
* TODO
\section report_making Making a Report Module
TODO
Refer to \ref mod_report_page for details on building a report module.
*/

2
Testing/test/qa-functional/src/org/sleuthkit/autopsy/testing/RegressionTest.java
View File

@@ -200,7 +200,7 @@ public class RegressionTest extends TestCase{
jbo0.pushNoBlock();
JFileChooserOperator jfco0 = new JFileChooserOperator();
jfco0.chooseFile(words);
JCheckBoxOperator jcbo = new JCheckBoxOperator(jdo, "Use during ingest", 0);
JCheckBoxOperator jcbo = new JCheckBoxOperator(jdo, "Enable for ingest", 0);
jcbo.doClick();
JButtonOperator jbo2 = new JButtonOperator(jdo, "OK", 0);
jbo2.pushNoBlock();

5
docs/doxygen/Doxyfile
View File

@@ -655,8 +655,9 @@ WARN_LOGFILE =
# directories like "/usr/src/myproject". Separate the files or directories
# with spaces.
INPUT = main.dox \
design.dox \
INPUT = main.dox workflow.dox \
modContent.dox modDev.dox modIngest.dox modReport.dox \
modResult.dox platformConcepts.dox \
regressionTesting.dox \
../../Case/src \
../../CoreComponentInterfaces/src \

87
docs/doxygen/design.dox
View File

@@ -1,87 +0,0 @@
/*! \page design_page General Design
\section design_overview Overview
This section outlines Autopsy design from the typical analysis work flow perspective.
This page is organized based on these phases:
- A Case is created.
- Images are added to the case and ingest modules are run.
- Results are manually reviewed and searched.
- Reports are generated.
\section design_case Creating a Case
The first step in Autopsy work flow is creating a case. This is done in the org.sleuthkit.autopsy.casemodule package (package \ref casemodule_overview). This module contains the wizards needed and deals with how to store the information. You should not need to do much modifications in this package. But, you will want to use the org.sleuthkit.autopsy.casemodule.Case object to access all data related to this case.
\section design_image Adding an Image
After case is created, one or more disk images can be added to the case. There is a wizard to guide that process
and it is located in the org.sleuthkit.autopsy.casemodule package. Refer to the package section \ref casemodule_add_image for more details on the wizard.
After image has been added to the case, the user can select one or more ingest modules to be executed on the image.
Ingest modules focus on a specific type of analysis task and run in the background. The results from the ingest module can be found in the results tree and in the ingest inbox.
The org.sleuthkit.autopsy.ingest package provides the basic infrastructure for the ingest module management. See the \ref ingestmodue_contents page for more details.
A list of standard modules that come with Autopsy can be found in:
- org.sleuthkit.autopsy.keywordsearch
- org.sleuthkit.autopsy.recentactivity
- org.sleuthkit.autopsy.hashdatabase
- org.sleuthkit.autopsy.thunderbirdparser
See \ref ingestmodule_making for more details on making an ingest module.
\section design_view Viewing Results
The UI has three main areas. The tree on the left-hand side, the result viewers in the upper right, and the content viewers in the lower right. Data passes between these areas by encapsulating them in Node objects (see org.openide.nodes.Node). Nodes use property sheets to encapsulate data (blackboard attributes) and are modeled in a parent-child hierarchy with other nodes.
The hierarchy is used to visually represent the data and to trigger child node updates when the parent node is selected.
Node child factories are invoked by the Netbeans framework at the time of parent node selection to populate and refresh the child node view.
The tree on the left hand-side shows the analysis results. Its contents are populated from the central database. See the org.sleuthkit.autopsy.directorytree module for more details.
The area in the upper right is the result viewer area. When a node is selected from the tree, the data is sent to this area. It is a framework with modules that display the data in different layouts. The org.sleuthkit.autopsy.corecomponentsinterfaces package has the interface to make one of these modules.
When an item is selected from the result viewer area, it is passed to the bottom right content viewers. It too is a framework with many modules that know how to show information about a specific file in different ways. The org.sleuthkit.autopsy.corecomponentsinterfaces package has the interface to make one of these modules. See XXX on building new content viewers.
<!-- @@@ MOVE THIS SOMEWHERE ELSE -- the directory tree package maybe??
The component is by default registered with the ingest manager as an ingest event listener.
The viewer first loads all the viewer-supported data currently in the blackboard when Autopsy starts.
During the ingest process the viewer receives events from ingest modules
(relayed by ingest manager) and it selectively refreshes parts of the tree providing real-time updates to the user.
When ingest is completed, the viewer responds to the final ingest data event generated by the ingest manager,
and performs a final refresh of all viewer-supported data in the blackboard.
User normally initiates result navigation in the directory tree.
Node content support capabilities are registered in the node's Lookup.
Multiple content viewers (such as strings, hex, extracted text, media) can support the node content.
If multiple content viewers are supported, a preferred (default) content viewer is chosen.
For more information refer to org.sleuthkit.autopsy.corecomponents, org.sleuthkit.autopsy.corecomponentsinterfaces
and
-->
\section design_report Report generation
When ingest is complete, the user can generate reports. There is a reporting framework to enable many different formats. Autopsy currently comes with generic html, xml and Excel reports. See the org.sleuthkit.autopsy.report package for details on the framework and
\ref report_making for details on building a new report module.
<!--Each reporting submodule implements org.sleuthkit.autopsy.report.ReportModule interface and registers itself in layer.xml
Reporting submodule typically interacts with 3 components:
- org.sleuthkit.autopsy.report.ReportConfiguration - to read current reporting configuration set by the user,
- Blackboard API in org.sleuthkit.datamodel.SleuthkitCase class - to traverse and read blackboard artifacts and attributes,
- an API (possibly external/thirdparty API) to convert blackboard artifacts data structures to the desired reporting format.
Please refer to report.dox and org.sleuthkit.autopsy.report package API documentation for more details on how to implement a custom reporting submodule.
-->
*/

9
docs/doxygen/main.dox
View File

@@ -3,9 +3,16 @@
<h3>Overview</h3>
Autopsy has been designed as a platform for open source tools besides just The Sleuth Kit. This document is for developers who want to add functionality into Autopsy. This could be in the form of enhancing the existing functionality or by making a module that plugs into it and you may distribute from your own site or push it back into the base distribution.
- \subpage design_page
- \subpage workflow_page
- \subpage platform_page
- \subpage mod_dev_page
- \subpage mod_ingest_page
- \subpage mod_report_page
- \subpage mod_content_page
- \subpage mod_result_page
- \subpage regression_test_page
<!-- Note that Case still has lots of content in package.dox -->
*/

33
docs/doxygen/modContent.dox Normal file
View File

@@ -0,0 +1,33 @@
/*! \page mod_content_page Developing Content Viewer Modules
Note that this needs cleanup and updating. it has been copied from the CoreComponentInterfaces package description.
<h3>Creating a DataContentViewer</h3>
DataContentTopComponent is the high-level window in the DataContent area. Each instance of this loads up all instances of DataContentViewers that have been registered with the system. Example viewers include the strings and hexdump views. If you want to make your own type of viewer, follow the steps below.
<ol>
<li>Create a module from within NetBeans. It must be dependent on these modules:
<ul>
<li>Case
<li>CoreComponentInterfaces
<li>CoreComponents
<li>DataModel
<li>DialogsAPI (if pop-ups and such are going to be used)
<li>Explorer & Property Sheet API
<li>Lookup
<li>Nodes API
<li>Setting API
<li>UI Utilities API
<li>Utilities API
<li>Window System API
</ul>
<li>Make a class that implements org.sleuthkit.autopsy.corecomponentinterfaces.DataContentViewer and is registered as a service provider for DataContentViewer.class by specifying "@ServiceProvider(service = DataContentViewer.class)" or by using layer.xml. This class must extend JPanel. </li>
</ol>
Find a home for this:
If multiple content viewers are supported, a preferred (default) content viewer is chosen.
*/

28
docs/doxygen/modDev.dox Normal file
View File

@@ -0,0 +1,28 @@
/*! \page mod_dev_page Development Overview
<!-- @@@ Update -->
\section mod_dev_setup Basic Setup
what do they need to download and configure to develop modules? Refer to BUILDING.txt as much as possible. Do they need to download source to develop modules or can they use a releae version?
\section mod_dev_module How to Create a Module
Steps to make a module (Netbeans menu options, adding in Autopsy as a platform, etc.
\section mod_dev_services Getting Access to Services
List the services and how to get access to them. Currently, I can think of Case, Loging, IngestManager.... The class description for each service class should have the details.
\section mod_dev_properties Saving Settings and Properties
If a module needs to have options and properties, how should they store them and get themsevles setup in the options viewer.
\section mod_dev_events Registering for events
What events hsould modules register for.. I'm not sure what goes in here with the new design.
- Case Updates?
We also need to sync this up with \ref ingestmodule_events.
*/

160
docs/doxygen/modIngest.dox Normal file
View File

@@ -0,0 +1,160 @@
/*! \page mod_ingest_page Developing Ingest Modules
\section ingestmodule_modules Ingest Module Basics
Ingest modules analyze data from a disk image. They typically focus on a specific type of data analysis. The modules are loaded each time that Autopsy starts. The user can choose to enable each module when they add an image to the case.
There are two types of ingest modules.
- Image-level modules are passed in a reference to an image and perform general analysis on it. These modules may query the database for a small set of files.
- File-level modules are passed in a reference to each file. The Ingest Manager chooses which files to pass and when. These modules are intended to analyze most of the files on the system or that want to examine the file content of all files (i.e. to detect file type based on signature instead of file extension).
Modules post their results to the blackboard (@@@ NEED REFERENCE FOR THIS -- org.sleuthkit.datamodel) and can query the blackboard to get the results of previous modules. For example, the hash database lookup module may want to query for a previously calculated hash value.
The IngestManager class is responsible for launching the ingest modules and passing data to them. Modules can send messages to the ingest inbox (REFERENCE) so that users can see when data has been found.
\section ingestmodule_making Making Ingest Modules
Refer to org.sleuthkit.autopsy.ingest.example for sample source code.
\subsection ingestmodule_making_api Module Interface
The first step is to choose the correct module type. Image-level modules will implement the IngestModuleImage interface and file-level modules will implement the IngestModuleAbstractFile interface.
There is a static getDefault() method that is not part of the interface, that every module (whether an image or a file module) needs to implement to return the registered static instance of the module. Refer to example code in example.ExampleAbstractFileIngestModule.getDefault()
File-level modules need to be singleton (only a single instance at a time). To ensure this, make the constructor private. Ensure the default public file module constructor is overridden with the private one. Image-level modules require a public constructor.
The interfaces have several standard methods that need to be implemented. See the interface methods for details.
- IngestModuleAbstract.init() is invoked every time an ingest session starts. A module should support multiple invocations of init() throughout the application life-cycle.
- IngestModuleAbstract.complete() is invoked when an ingest session completes. The module should perform any resource (files, handles, caches) cleanup in this method and submit final results and post a final ingest inbox message.
- IngestModuleAbstract.stop() is invoked on a module when an ingest session is interrupted by the user or by the system.
The method implementation should be similar to complete() in that the module should perform any cleanup work. If there is pending data to be processed or pending results to be reported by the module then the results should be rejected and ignored if stop() is invoked and the module should terminate as early as possible.
- process() method is invoked to analyze the data. The specific method depends on the module type.
Multiple images can be ingested at the same time. The current behavior is that the files from the second image are added to the list of the files from the first image. The impact of this on module development is that a file-level module could be passed in files from different images in consecutive calls to process(). New instances of image-level modules will be created when the second image is added. Therefore, image-level modules should assume that the process() method will be called only once after init() is called.
Every module should support multiple init() - process() - complete(), and init() - process() - stop() invocations.
The modules should also support multiple init() - complete() and init() - stop() invocations,
which can occur if ingest pipeline is started but no work is enqueued for the particular module.
Module developers are encouraged to use the standard java.util.logging.Logger infrastructure to log errors to the Autopsy log.
\subsection ingestmodule_making_process Process Method
The process method is where the work is done in each type of module. Some notes:
- File-level modules will be called on each file in an order determined by the IngestManager. Each module is free to quickly ignore a file based on name, signature, etc. If a module wants to know the return value from a previously run module, it should use the IngestServices.getAbstractFileModuleResult() method.
- Image-level modules are expected not passed in specific files and are expected to query the database to find the files that they are interested in.
\subsection ingestmodule_making_registration Module Registration
Ingest modules need to be registered using the Netbeans Lookup infrastructure in package's layer.xml file.
An example Image-level module is:
\verbatim
<file name="org-sleuthkit-autopsy-ingest-example-ExampleImageIngestModule.instance">
<attr name="instanceOf" stringvalue="org.sleuthkit.autopsy.ingest.IngestModuleImage"/>
<attr name="instanceCreate" methodvalue="org.sleuthkit.autopsy.ingest.example.ExampleImageIngestModule.getDefault"/>
<attr name="position" intvalue="1000"/>
</file>
\endverbatim
An example file-level module is:
\verbatim
<file name="org-sleuthkit-autopsy-ingest-example-ExampleAbstractFileIngestModule.instance">
<attr name="instanceOf" stringvalue="org.sleuthkit.autopsy.ingest.IngestModuleAbstractFile"/>
<attr name="instanceCreate" methodvalue="org.sleuthkit.autopsy.ingest.example.ExampleAbstractFileIngestModule.getDefault"/>
<attr name="position" intvalue="1100"/>
</file>
\endverbatim
Note the "position" attribute. The attribute determines the ordering of the module in the ingest pipeline.
Modules with lower position attribute will execute earlier.
Use high numbers (higher than 1000) for non-core modules. If your module depends on results from another module, use a higher position attribute to enforce the dependency.
Note: we plan to implement a more flexible and robust module dependency system in future versions of the Autopsy ingest framework.
New modules can be added to the Autopsy ingest pipeline by dropping in jar files into build/cluster/modules.
Dropped in module will be automatically recognized next time Autopsy starts.
\subsection ingestmodule_making_results Posting Results
<!-- @@@ -->
NOTE: This needs to be made more in sync with the \ref platform_blackboard and \ref platform_inbox sections.
Users will see the results from ingest modules in one of two ways:
- Results are posted to the blackboard and will be displayed in the navigation tree
- Messages are sent to the Ingest Inbox to notify a user of what has recently been found.
See the Blackboard (REFERENCE) documentation for posting results to it. Modules are free to immediately post results when they find them or they can wait. The IngestServices.getUpdateFrequency() method returns the maximum amount of time that a module can wait before it posts its results.
An example of waiting to post results is the keyword search module. It is resource intensive to commit the keyword index and do a keyword search. Therefore, when its process() method is invoked, it checks if it is close to the getUpdateFrequency() since the last time it did a keyword search. If it is, then it commits the index and performs the search.
When they add data to the blackboard, modules should notify listeners of the new data by periodically invoking IngestServices.fireModuleDataEvent() method. This allows other modules (and the main UI) to know when to query the blackboard for the latest data.
Modules should post messages to the inbox when interesting data is found. The messages includes the module name, message subject, message details, a unique message id (in the context of the originating module), and a uniqueness attribute. The uniqueness attribute is used to group similar messages together and to determine the overall importance priority of the message (if the same message is seen repeatedly, it is considered lower priority).
It is important though to not fill up the inbox with messages. These messages should only be sent if the result has a low false positive rate and will likely be relevant. For example, the hash lookup module will send messages if known bad (notable) files are found, but not if known good (NSRL) files are found. The keyword search module will send messages if a specific keyword matches, but will not send messages (by default) if a regular expression match for a URL has matches (because a lot of the URL hits will be false positives and can generate thousands of messages on a typical system).
Ingest messages have different types: there are info messages, warning messages, error messages and data messages.
The data messages contain encapsulated blackboard artifacts and attributes. The passed in data is used by the ingest inbox GUI widget to navigate to the artifact view in the directory tree, if requested by the user.
Ingest message API is defined in IngestMessage class. The class also contains factory methods to create new messages.
Messages are posted using IngestServices.postMessage() method, which accepts a message object created using one of the factory methods.
Modules should post inbox messages to the user when stop() or complete() is invoked (refer to the examples).
It is recommended to populate the description field of the complete inbox message to provide feedback to the user
summarizing the module ingest run and if any errors were encountered.
\subsection ingestmodule_making_configuration Module Configuration
<!-- @@@ -->
NOTE: Make sure we update this to reflect \ref mod_dev_properties and reduce duplicate comments.
Ingest modules may require user configuration. The framework
supports two levels of configuration: run-time and general. Run-time configuration
occurs when the user selects which ingest modules to run when an image is added. This level
of configuration should allow the user to enable or disable settings. General configuration is more in-depth and
may require an interface that is more powerful than simple check boxes.
As an example, the keyword search module uses both configuration methods. The run-time configuration allows the user
to choose which lists of keywords to search for. However, if the user wants to edit the lists or create lists, they
need to do go the general configuration window.
Module configuration is decentralized and module-specific; every module maintains its
own configuration state and is responsible for implementing the graphical interface.
The run-time configuration (also called simple configuration), is achieved by each
ingest module providing a JPanel. The IngestModuleAbstract.hasSimpleConfiguration(),
IngestModuleAbstract.getSimpleConfiguration(), and IngestModuleAbstract.saveSimpleConfiguration()
methods should be used for run-time configuration.
The general configuration is also achieved by the module returning a JPanel. A link will be provided to the general configuration from the ingest manager if it exists.
The IngestModuleAbstract.hasAdvancedConfiguration(),
IngestModuleAbstract.getAdvancedConfiguration(), and IngestModuleAbstract.saveAdvancedConfiguration()
methods should be used for general configuration.
\section ingestmodule_events Getting Ingest Status and Events
<!-- @@@ -->
NOTE: Sync this up with \ref mod_dev_events.
Other modules and core Autopsy classes may want to get the status of the ingest manager. The IngestManager provides access to this data with the sleuthkit.autopsy.ingest.IngestManager.isIngestRunning() method.
External modules can also register themselves as ingest module event listeners and receive event notifications (when a module is started, stopped, completed or has new data).
Use the IngestManager.addPropertyChangeListener() method to register a module event listener.
Events types received are defined in IngestManager.IngestModuleEvent enum.
At the end of the ingest, IngestManager itself will notify all listeners of IngestModuleEvent.COMPLETED event.
The event is an indication for listeners to perform the final data refresh by quering the blackboard.
Module developers are encouraged to generate periodic IngestModuleEvent.DATA
ModuleDataEvent events when they post data to the blackboard,
but the IngestManager will make a final event to handle scenarios where the module did not notify listeners while it was running.
*/

12
docs/doxygen/modReport.dox Normal file
View File

@@ -0,0 +1,12 @@
/*! \page mod_report_page Developing Report Modules
<!-- @@@ Cleanup -->
Each reporting submodule implements org.sleuthkit.autopsy.report.ReportModule interface and registers itself in layer.xml
Reporting submodule typically interacts with 3 components:
- org.sleuthkit.autopsy.report.ReportConfiguration - to read current reporting configuration set by the user,
- Blackboard API in org.sleuthkit.datamodel.SleuthkitCase class - to traverse and read blackboard artifacts and attributes,
- an API (possibly external/thirdparty API) to convert blackboard artifacts data structures to the desired reporting format.
*/

31
docs/doxygen/modResult.dox Normal file
View File

@@ -0,0 +1,31 @@
/*! \page mod_result_page Developing Result Viewer Modules
NOTE: This has been moved from a package-level description and needs cleanup.
<h3>Creating a DataResultViewer</h3>
<p>DataResultTopComponent is the high-level window in the DataResult area. Each instance of this loads up all instances of DataResultViewers that have been registered with the system. Example viewers include the table and thumbnail views. If you want to make your own type of viewer, follow the steps below.
<ol>
<li>Create a module from within NetBeans. It must be dependent on these modules:
<ul>
<li>Case
<li>CoreComponentInterfaces
<li>CoreComponents
<li>DataModel
<li>DialogsAPI (if pop-ups and such are going to be used)
<li>Explorer & Property Sheet API
<li>Lookup
<li>Nodes API
<li>Setting API
<li>UI Utilities API
<li>Utilities API
<li>Window System API
</ul>
<li>Make a class that extends org.sleuthkit.autopsy.corecomponents.AbstractDataResultViewer and is registered as a service provider for the org.sleuthkit.autopsy.corecomponentinterfaces.DataResultViewer class by specifying "@ServiceProvider(service = DataResultViewer.class)" or by using layer.xml. This class will extend JPanel. </li>
<li>See the previous sections on default actions. (note that this refers to the CoreComponentINterfaces package-level description, which I think is now in \ref design_data_flow).</li>
</ol>
*/

20
docs/doxygen/platformConcepts.dox Normal file
View File

@@ -0,0 +1,20 @@
/*! \page platform_page Platform Concepts
\section platform_basics Basic Concepts
- Central database
- Ingest Modules that analyze data quickly and provide quick results
- Single tree to browse for results
- Framework to view sets of results with (table, thumbnail, etc.)
- Framework to view file content (hex, strings, etc.)
\section platform_blackboard The Blackboard
Provide some basic details, link to the C++ docs, and link to the Java API.
\section platform_inbox Ingest Inbox
Provide some details and API
*/

93
docs/doxygen/workflow.dox Normal file
View File

@@ -0,0 +1,93 @@
/*! \page workflow_page General Workflow and Design
\section design_overview Overview
This section outlines Autopsy design from the typical analysis work flow perspective.
This page is organized based on these phases:
- A Case is created.
- Images are added to the case and ingest modules are run.
- Results are manually reviewed and searched.
- Reports are generated.
\section design_case Creating a Case
The first step in Autopsy work flow is creating a case. This is done in the org.sleuthkit.autopsy.casemodule package (see \ref casemodule_overview for details). This module contains the wizards needed and deals with how to store the information. You should not need to do much modifications in this package. But, you will want to use the org.sleuthkit.autopsy.casemodule.Case object to access all data related to this case.
\section design_image Adding an Image and Running Ingest Modules
After case is created, one or more disk images can be added to the case. There is a wizard to guide that process and it is located in the org.sleuthkit.autopsy.casemodule package. Refer to the package section \ref casemodule_add_image for more details on the wizard. Most developers will not need to touch this code though.
After image has been added to the case, the user can select one or more ingest modules to be executed on the image. Ingest modules focus on a specific type of analysis task and run in the background. The results from the ingest module can be found in the results tree and in the ingest inbox.
The org.sleuthkit.autopsy.ingest package provides the basic infrastructure for the ingest module management.
A list of standard ingest modules that come with Autopsy can be found in:
- org.sleuthkit.autopsy.keywordsearch
- org.sleuthkit.autopsy.recentactivity
- org.sleuthkit.autopsy.hashdatabase
- org.sleuthkit.autopsy.thunderbirdparser
See \ref mod_ingest_page for more details on making an ingest module.
\section design_view Viewing Results
The UI has three main areas. The tree on the left-hand side, the result viewers in the upper right, and the content viewers in the lower right. Data passes between these areas by encapsulating them in Netbeans Node objects (see org.openide.nodes.Node). These allow Autopsy to generically handle all types of data. The org.sleuthkit.autopsy.datamodel package details with wrapping the org.sleuthkit.datamodel objects as Netbeans Nodes.
Nodes are modeled in a parent-child hierarchy with other nodes. All data within a Case is represented in a hierarchy with the disk images being one level below the case and volumes and such below the image.
The tree on the left hand-side shows the analysis results. Its contents are populated from the central database. This is where you can browse the file system contents and see the results from the blackboard (see \ref blackboard_page). The tree is implemented in the org.sleuthkit.autopsy.directorytree package.
The area in the upper right is the result viewer area. When a node is selected from the tree, the node and its children are sent to this area. This area is used to view a set of nodes. The viewer is itself a framework with modules that display the data in different layouts. For example, the standard version comes with a table viewer and a thumbnail viewer. Refer to \ref mod_result_page for details on building a module.
When an item is selected from the result viewer area, it is passed to the bottom right content viewers. It too is a framework with many modules that know how to show information about a specific file in different ways. For example, there are viewers that show the data in a hex dump format, extract the strings, and display pictures and movies.
See \ref mod_content_page for details on building new content viewers.
\section design_report Report generation
When ingest is complete, the user can generate reports. There is a reporting framework to enable many different formats. Autopsy currently comes with generic html, xml and Excel reports. See the org.sleuthkit.autopsy.report package for details on the framework and
\ref report_making for details on building a new report module.
<!-- @@@ MOVE THIS SOMEWHERE ELSE -- the directory tree package maybe??
The component is by default registered with the ingest manager as an ingest event listener.
The viewer first loads all the viewer-supported data currently in the blackboard when Autopsy starts.
During the ingest process the viewer receives events from ingest modules
(relayed by ingest manager) and it selectively refreshes parts of the tree providing real-time updates to the user.
When ingest is completed, the viewer responds to the final ingest data event generated by the ingest manager,
and performs a final refresh of all viewer-supported data in the blackboard.
Node content support capabilities are registered in the node's Lookup.
-->
<!-- @@@ This is too detailed for here, but maybe should be broken up and put into the sections on making a result viewer and such…
\section design_data_flow Data Flow
\subsection design_data_flow_create Creating Nodes in DataExplorer
Data flows between the UI zones using a NetBeans node. The DataExplorer modules create the NetBeans nodes. They query the SQLite database or do whatever they want to identify the set of files that are of interest. They create the NetBeans nodes based on Sleuthkit data model objects. See the org.sleuthkit.autopsy.datamodel package for more details on this.
\subsection design_data_flow_toResult Getting Nodes to DataResult
Each DataExplorer TopComponent is responsible for creating its own DataResult TopComponent to display its results. It can choose to re-use the same TopComponent for multiple searches (as DirectoryTree does) or it can choose to make a new one each time (as FileSearch does). The setNode() method on the DataResult object is used to set the root node to display. A dummy root node must be created as the parent if a parent does not already exist.
The DataExplorer is responsible for setting the double-click and right-click actions associated with the node. The default single click action is to pass data to DataContent. To override this, you must create a new DataResultViewer instance that overrides the propertyChange() method. The DataExplorer adds actions to wrapping the node in a FilterNode variant. The FilterNode then defines the actions for the node by overriding the getPreferredAction() and getActions() methods. As an example, org.sleuthkit.autopsy.directorytree.DataResultFilterNode and org.sleuthkit.autopsy.directorytree.DataResultFilterChildren wraps the nodes that are passed over by the DirectoryTree DataExplorer.
DataResult can send data back to its DataExplorer by making a custom action that looks up it's instance (DataExplorer.getInstance()).
\subsection design_data_flow_toContent Getting Nodes to DataContent
A default DataContent viewer is created when a case is opened. To display the contents of a node, it must be passed to a DataContent instance. The default single-click behavior of the DataResultViewers is to lookup the default DataContent TopComponent and pass the selected node to it. See org.sleuthkit.autopsy.corecomponents.AbstractDataResultViewer.propertyChange(PropertyChangeEvent) for details.
-->
*/