Ghidra is a software reverse engineering (SRE) framework
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="content-type">
<title>Language Versioning and Migration</title>
<style name="text/css">
li { font-family:times new roman; font-size:14pt; margin-bottom: 12px; }
h1 { color:#000080; font-family:times new roman; font-size:36pt; font-style:italic; font-weight:bold; text-align:center; }
h2 { padding-top:30px; color:#984c4c; font-family:times new roman; font-size:18pt; font-weight:bold; }
h2 { color:#984c4c; font-family:times new roman; font-size:18pt; font-weight:bold; color:#984c4c; }
h3 { color:#0000ff; font-family:times new roman; font-size:14pt; font-weight:bold; color:#0000ff; }
p { font-family:times new roman; font-size:14pt; }
td { font-family:times new roman; font-size:14pt; padding-left:10px; padding-right:10px; vertical-align: top; }
th { font-family:times new roman; font-size:14pt; font-weight:bold; padding-left:10px; padding-right:10px; }
</style>
</head>
<body>
<h1>Language Versioning and
Migration</h1>
<p>This document discusses the mechanisms within Ghidra which mitigate
the impact of language modifications to existing user program
files. There are two general classes of language modifications
which can be supported by the language translation capabilities within
Ghidra :<br>
</p>
<ol>
<li><a href="#versioning"><span style="font-weight: bold;">Version
Change</span></a> - caused by modifications to a specific
language implementation which necessitates a re-disassembly of a programs
instructions.</li>
<li><a href="#forcedMigration"><span style="font-weight: bold;">Forced
Language Migration</span></a> - caused when an existing language
implementation is completely replaced by a new implementation (language
name must be different). It is important that an "old" language
file (<span style="font-style: italic;">*.lang</span>) be generated for
a language before it is eliminated. A simple or custom
language translator is required to facilitate the forced migration.<br>
</li>
</ol>
<p>Any program opened within Ghidra whose language has had a version
change or has been replaced by a new implementation will be forced to
upgrade. This will prevent such a program file from being opened
as immutable and will impose a delay due to the necessary
re-disassembly of all instructions.</p>
<p>In addition to a forced upgrade, Ghidra's <span
style="font-style: italic;">Set Language</span> capability will allow
a user to make certain transitions between similar language
implementations. Such transitions are generally facilitated via a
default translator, although certain limitations are imposed based upon
address space sizes and register mappings.</p>
<p>Any changes made to a Data Organization could impact the packing of
components within a structure or union. While such changes should be avoided
due to the possible fallout, any such change to a
<span style="font-style: italic;">*.cspec</span> should be made
in conjunction with a version change to all affected languages within
the relevant <span style="font-style: italic;">*.ldefs</span> files. The
resulting program upgrade will allow affected data types to be updated.
<br>
</p>
<h2><a name="versioning"></a>Language Versioning</h2>
<p>A language's version is specified as a<span
style="font-style: italic;"> <major></span><span
style="font-weight: bold; font-style: italic;">.</span><span
style="font-style: italic;"><minor></span> number pair (e.g.,
1.0). The decision to advance the major or minor version number
should be based upon the following criteria:<br>
</p>
<ol>
<li><span style="font-weight: bold;">Major Version Change</span> -
caused by modifications to a specific
language implementation which changes register addressing or context
register schema. Addition of registers alone does not constitute
a major or minor change.<br>
</li>
<li><span style="font-weight: bold;">Minor Version Change</span> -
caused by modifications to a specific
language implementation which changes existing instruction or
subconstructor pattern matching. Pcode changes and addition of
new instructions alone does not constitute a major or minor change.</li>
</ol>
<p>Anytime the major version number is advanced, the minor version
number should be reset to zero. <br>
</p>
<p>Only major version changes utilize a <a href="#translators">Language
Translator</a> to facilitate the language transition.<br>
</p>
<h2><a name="forcedMigration"></a>Forced Language Migration<br>
</h2>
<p>When eliminating an old language the following must be accomplished:<br>
</p>
<ol>
<li>Establish a replacement language</li>
<li>Generate <a href="#oldlang">old-language specification</a> file (<span
style="font-style: italic;">*.lang</span>)<br>
</li>
<li>Establish one and only one <a href="#translators">Language
Translator </a>from the final version of the eliminated language to
its replacement language.<br>
</li>
</ol>
<p>Before eliminating a language a corresponding "old" language file
must be generated and stored somewhere within Ghidra's languages
directory (<span style="font-style: italic;">core/languages/old</span>
directory has been established for this purpose). In
addition, a simple or custom <a href="#translators">Language Translator</a>
must be established to facilitate the language migration to the
replacement language. <br>
</p>
<p>An old-language file may be generated automatically while the
language still exists using the <span
style="font-style: italic; font-weight: bold;">GenerateOldLanguagePlugin
</span>configured into Ghidra's project window. In addition, if
appropriate, a draft simple Language Translator specification can
generated provided the replacement language is also available.<br>
</p>
<p>To generate an old-language file and optionally a draft simple
translator specification:<br>
</p>
<ul>
<li>Choose the menu item File>Generate Old Language File...</li>
<li>Select the language to be eliminated from the languages list and
click <span style="font-weight: bold;">Generate...</span></li>
<li>From the file chooser select the output directory, enter a
suitable name for the file and click <span style="font-weight: bold;">Create</span></li>
<li>Once the old-language file is generated you will be asked if you
would like to <span style="font-weight: bold; font-style: italic;">Create
a Simple Translator?</span> If the replacement language is
complete and available you can click <span style="font-weight: bold;">Yes
</span>and specify an output file with the file chooser.</li>
</ul>
<h2><a name="oldlang"></a>Old Language Specification (<span
style="font-style: italic;">*.lang</span>)</h2>
<p>An old-language specification file is used to describe the essential
elements of a language needed to instantiate an old program using that
language and to facilitate translation to a replacement language.<br>
</p>
<p>The specification file is an XML file which identifies a language's
description, address spaces and named registers. Since it
should be generated using the <span
style="font-style: italic; font-weight: bold;">GenerateOldLanguagePlugin</span>,
its syntax is not defined here.
</p>
<p><span style="font-style: italic; font-weight: bold;">Sample
Old-Language Specification File:</span><br>
</p>
<pre><?xml version="1.0" encoding="UTF-8"?><br><language version="1" endian="big"><br> <description><br> <name>MyOldProcessorLanguage</name><br> <processor>MyOldProcessor</processor><br> <family>Motorola</family><br> <alias>MyOldProcessorLanguageAlias1</alias><br> <alias>MyOldProcessorLanguageAlias2</alias><br> </description><br> <spaces><br> <space name="ram" type="ram" size="4" default="yes" /><br> <space name="register" type="register" size="4" /><br> <space name="data" type="code" size="4" /><br> </spaces><br> <registers><br> <context_register name="contextreg" offset="0x40" bitsize="8"><br> <field name="ctxbit1" range="1,1" /><br> <field name="ctxbit0" range="0,0" /><br> </context_register><br> <register name="r0" offset="0x0" bitsize="32" /><br> <register name="r1" offset="0x4" bitsize="32" /><br> <register name="r2" offset="0x8" bitsize="32" /><br> <register name="r3" offset="0xc" bitsize="32" /><br> <register name="r4" offset="0x10" bitsize="32" /><br> </registers><br></language><br><br></pre>
<h2><a name="translators"></a>Language Translators</h2>
<p>A language translator facilitates the renaming of address spaces,
and relocation/renaming of registers. In addition, stored
register values can be transformed - although limited knowledge is
available for decision making. Through the process of
re-disassembly, language changes in instruction and subconstructor
pattern matching is handled. Three forms of translators are
supported:<br>
</p>
<ol>
<li><span style="font-weight: bold;">Default Translator</span> - in
the absence of a simple or custom translator, an attempt will be made
to map all address spaces and registers. Stored register values
for unmapped registers will be discarded. Forced language
migration can not use a default translator since it is the presence of
a translator with an old-language which specifies the migration path.<br>
</li>
<li><span style="font-weight: bold;">Simple Translator</span> -
extends the behavior of the default translator allowing specific
address space and register mappings to be specified via an XML file (<span
style="font-style: italic;">*.trans</span>). See sample <a
href="#transfile">Simple Translator Specification</a>.<br>
</li>
<li><span style="font-weight: bold;">Custom Translator</span> -
custom translators can be written as a Java class which extends <span
style="font-weight: bold; font-style: italic;">LanguageTranslatorAdapter</span>
or implements <span style="font-weight: bold; font-style: italic;">LanguageTranslator</span>.
This should generally be unnecessary but can provided additional
flexibility. The default constructor must be public and will be used
for instantiation. Extending <span
style="font-weight: bold; font-style: italic;">LanguageTranslatorAdapter
</span>will allow the default translator capabilities to be
leveraged with minimal coding.</li>
</ol>
<span style="font-family: times new roman;"><span
style="font-weight: bold;"></span></span>
<p style="font-weight: bold; font-style: italic;"><a name="transfile"></a>Sample
Simple Translator Specification File:</p>
<pre><?xml version="1.0" encoding="UTF-8"?>
<language_translation>
<from_language version="1">MyOldProcessorLanguage</from_language>
<to_language version="1">MyNewProcessorLanguage</to_language>
<!--
Obsolete space will be deleted with all code units in that space.
-->
<delete_space name="data" />
<!--
Spaces whose name has changed can be mapped over
-->
<map_space from="ram" to="ram" />
<!--
Registers whose name has changed can be mapped (size and offset changes are allowed)
The map_register may include a size attribute although it is ignored.
-->
<map_register from="r0" to="cr0" />
<map_register from="r1" to="cr1" />
<!--
All existing processor context can be cleared
-->
<clear_all_context/>
<!--
A specific context value can be painted across all of program memory
NOTE: sets occur after clear_all_context
-->
<set_context name="ctxbit0" value="1"/>
<!--
Force a specific Java class which extends
<i>ghidra.program.util.LanguagePostUpgradeInstructionHandler</i>
to be invoked following translation and re-disassembly to allow for more
complex instruction context transformations/repair.
-->
<post_upgrade_handler class="ghidra.program.language.MyOldNewProcessorInstructionRepair" />
</language_translation>
</pre>
<p style="font-weight: bold; font-style: italic;">Translator Limitations<br>
</p>
<p>The current translation mechanism does not handle the potential need
for complete re-disassembly and associated auto-analysis. <br>
</p>
</body>
</html>