 |
|
In Chapter 6, a taxonomy was developed for classifying computer and network attacks. This taxonomy was used in subsequent chapters to classify and analyze the Internet incidents reported to the CERT®/CC from 1998 to 1995. This chapter presents a brief critique of the taxonomy based on this experience. This is followed by a discussion of how incidents can be classified by using this taxonomy and other data from the incidents. 13.1. Review of the Characteristics of Satisfactory Taxonomies A taxonomy is an approximation of reality that is used to gain greater understanding in a field of study. Because it is an approximation, it will fall short in some characteristics. This may be particularly the case when the characteristics of the data being classified are imprecise and uncertain, as was the data for this study. Nevertheless, classification is an important and necessary process for systematic study. As presented in Chapter 6, a taxonomy should have classification categories with the following characteristics [Amo94:34]: 1) mutually exclusive - classifying in one category excludes all others because categories do not overlap, 2) exhaustive - taken together, the categories include all possibilities, 3) unambiguous - clear and precise so that classification is not uncertain, regardless of who is classifying, 4) repeatable - repeated applications result in the same classification, regardless of who is classifying, 5) accepted - logical and intuitive so that they could become generally approved, 6) useful - can be used to gain insight into the field of inquiry. These characteristics can be used to evaluate possible taxonomies. This will be done in the remaining sections of this chapter. 13.2. Evaluation of the taxonomy relative to the taxonomy criteria The following sections compare the taxonomy to each of the desired characteristics using the experience of applying the taxonomy in this research. It should be emphasized, however, that there is a difference between classifying an incident and an attack. The CERT®/CC records were of incidents, which were composed of numerous attacks (a distinction made in Chapters 1, 6, 7 and 12). This taxonomy only produces a single classification for single attacks. The remainder of this section evaluates the taxonomy for classifying attacks. Section 13.3 discusses using this taxonomy, along with other criteria, to classify an incident. This is more difficult since an incident can be made up of multiple attacks. 13.2.1. Categories that are Mutually Exclusive - The categories of a taxonomy should be such that classification into one category excludes classification into all others, because the taxonomy categories do not overlap. Care was taken in developing the categories of this taxonomy to ensure they were mutually exclusive (see Figure 6.9). In general, in applying the taxonomy for this research, there were few instances when a single classification was not directly determined. There were, however, two problems noted with respect to the categories being mutually exclusive. The first problem was that sometimes, when there was limited information, the determination of a single category was difficult. The second problem was that sometimes, one attack could theoretically be in two categories. An example of the first problem came in the vulnerability categories. Generally, the CERT®/CC records reported vulnerabilities in terms of software processes or programs. An example is the keyword sendmail. This key word indicated a vulnerability in sendmail was exploited, but it did not indicate whether this vulnerability resulted from an implementation, design or configuration error. More information would generally point to one category, although sometimes there could be a disagreement when there was not general acceptance of the definition of terms. This problem will be discussed in Section 13.2.5. The second problem noted with the categories being mutually exclusive was that one attack could theoretically be classified into two categories. This was particularly the case when differentiating between results and objectives. In these cases, the problem was that one attack could have multiple results, or accomplish multiple objectives. This was generally not a problem for the application of the taxonomy for this research, but then there were also few data in the CERT®/CC records about results and objectives (see Chapters 7 and 8). An example of a possible problem with objectives not being mutually exclusive might be in the destruction of files by one company on a rival company's computer system. In this case, damage was caused, and financial gain may be achieved (two possible objectives). On closer examination, however, the categories are found to be mutually exclusive. In this example, the objective should be classified as being financial gain. The damage to files should be classified in the corruption of files category of Results, which leads the attacker to the objective of financial gain. The greatest potential classification problem was with denial-of-service attacks. For example, if selected files in a user's account are deleted, then an argument could be made that both corruption of files and denial-of-service resulted (two results). In the application of the taxonomy for this research, an incident like this was classified in the corruption of information category. On the other hand, if the only files deleted were systems programs or files, such as the system's password file, the system's login program, or a user's account, then this incident was classified in the denial-of-service category. A more serious problem occurs if an attacker deleted all files on a system. In this case, there are clearly two results: destruction of data files (corruption of information), and destruction of system files (denial-of-service). Even knowing the attacker's motivation may not help to classify such an attack. The attacker's motivation may include both results ("I'm going to get the company that fired me by destroying all their records and shutting down their system so nobody can use it….."). This ambiguity concerning denial-of-service results was not a problem for this research, because it was generally obvious what category the attack should be classified in. On the other hand, the experience with this research showed that it could be a problem. This problem could be mitigated with more information, which would make the classifications easier. One final example of possible problems with categories being mutually exclusive was seen in the tools category (see Figure 6.9). With the exception of the data tap category, each of the tool categories may contain the other tool categories within them. For example, toolkits contain scripts, programs, and sometimes autonomous agents. So when a toolkit is used, the scripts and programs category is also included. User commands also must be used for the initiation of scripts, programs, autonomous agents, toolkits and distributed tools. In other words, there is an order to the categories in the tools block, from the simple user command category to the more sophisticated distributed tools category (the last category, data taps, is not related to the other categories). This is unlike the other blocks of the taxonomy. What made these categories in the tools block mutually exclusive when applied to the CERT®/CC records was that attacks were classified according to the highest category of tool used. 13.2.2. Categories that are Exhaustive - Taken together, the categories in a taxonomy should include all possibilities. In terms of the taxonomy developed for this research, all paths connecting attackers with objectives (see Figure 6.9) should be included. During the classification of the data for this research, there was no instance when a category could not be found for the data. With respect to the classification of the CERT®/CC data, the attackers and objectives blocks were exhaustive. There were such few data in these blocks, however, that there remains a question as to whether more categories would be necessary when classifying a larger data set. One question did arise for those instances when former employees were identified in the records as attackers. The categories in the attackers block were established according the motivations of the attackers and not who there are. For this research, former employees were classified as vandals. More questions may arise when attackers from a larger data set are classified. Another group of incidents that may lead to further refinements of categories in the taxonomy is internal attackers, where an attacker is located within the organization being attacked. The CERT®/CC incidents primarily involved external attackers, where the attackers were outside the organization being attacked. As such, the taxonomy is largely untested against incidents involving internal attackers. There were two adjustments made to the taxonomy during the research to add categories. The first instance was to add the data in transit category to the access block of the taxonomy. Originally, data in transit was considered to be in the files category, but it made more logical sense to separate it out because the data are in different forms when they are in a file or in transit across a network. In addition, the methods used for attack against files and data in transit may be different, which makes it important to have separate categories in the taxonomy. The second adjustment made to the taxonomy was to add the distributed tool and data tap categories to the tools block. As noted in Chapter 8, there were no instances of the use of these tools being recorded in the CERT®/CC records during the period of this research. The categories were added, however, to make sure the tools block was exhaustive. In the case of distributed tools, this was because of incidents recorded at the CERT®/CC in 1996 and 1997, after the period of this research. The data tap category is based on theoretical attacks that have not been recorded in any CERT®/CC incidents. This experience is an example of what we should expect, over time, regarding any taxonomy of computer and network attacks: the need will arise to add new categories. If nothing else, attackers will try new tools that may not be able to be classified into the current tools category. Greater understanding of attackers and their methods, however, may also make changes to other categories. 13.2.3. Categories that are Unambiguous - Categories in a taxonomy should be clear and precise so that classification is certain, regardless of who is classifying. Only one person did classifications for this research. As such, no determination was made as to whether classifications using the taxonomy were unambiguous. On the other hand, there were some difficulties in making classifications due to ambiguity. This was primarily the result of the lack of information. A typical example of this was a narrative description of an attack that did not contain enough information to classify the incident. If the classifications of attacks were done with more information available, such as by CERT®/CC personnel during an incident, ambiguity would be reduced. This is one course of action discussed in Chapter 14. 13.2.4. Categories that are Repeatable - Repeated applications of a taxonomy should result in the same classification, regardless of who is classifying. For this research, classifications of data were made only one time. As such, no determination was made as to whether classifications using the taxonomy were repeatable. On the other hand, it is likely that some of the classifications are, in fact, not repeatable because of incomplete information as described in the previous section. This is because, the greater the uncertainty when making a classification, the greater the chance for error or for disagreement in a classification. 13.2.5. Categories that are Accepted - The categories of a taxonomy should be logical and intuitive so that they could become generally approved. The taxonomy developed for this research was based upon a logical process that was intended to be intuitive. How logical and intuitive the taxonomy is could be investigated by having the taxonomy evaluated by others, such as during use by response personnel, as recommended in Chapter 14. One of the ways the taxonomy was designed to be widely accepted was in the use of simple and accepted terms for the classifications. Terms that were widely used, but which had controversial definitions were intentionally avoided. For example, the term computer virus is widely used, but there is no accepted definition. One set of terms that has some problems with accepted definitions that were necessary to include in the taxonomy were the three categories of vulnerabilities. An example of the problem with these terms is when a program such as sendmail is targeted with a mail spam attack (repeated mailings). If this causes a system's storage capacity to be exceeded, then service may be denied to users. Some would view the failure to check for too many messages as an implementation vulnerability because the person that implemented the sendmail code did not include the proper checks. On the other hand, others may view this as a vulnerability resulting from an improper design if its inclusion was not part of the design. Such problems were not seen in the application of this taxonomy to the CERT®/CC records. They can be avoided in the future by having good information about what is being classified, and by the use of specific definitions for the terms describing each category. As was noted earlier, not enough information was generally available for classification into the three categories of vulnerabilities, so, for this category of the access block, its application is largely untested. For other categories, however, no problems were indicated concerning the acceptance or the intuitive nature of any of the terms. 13.2.6. Categories that are Useful - The final characteristic of a satisfactory taxonomy is that it can be used to gain insight into the field of inquiry. The conclusions from this research were largely drawn from analyzing data that had been classified using the taxonomy of attacks presented in Chapter 6. This showed the usefulness of the taxonomy because the analysis could not have been conducted without such a taxonomy for classification. As discussed in Chapter 6, previous taxonomies were inadequate for this classification because they did not meet the criteria for a satisfactory taxonomy. As discussed in Chapter 6 (Section 6.2), the taxonomy is also potentially useful because it can organize thinking about computer and network security. The taxonomy emphasizes that, in order to be successful, an attacker must find one or more paths that connect the attackers to their objective. As the formal definition presented in Chapter 5 indicates, computer security is preventing attackers from achieving objectives by preventing them from making any complete connections through the process depicted in the taxonomy. More specifically, computer security efforts are aimed at the six blocks of the taxonomy. This is potentially useful because it helps direct policies and programs against specific targets or events in the attack process. Section 6.4.5 discussed how this might be done for each of the six blocks of the taxonomy. 13.3. Classifications of Incidents Classification of incidents is more difficult than the classification of attacks. First of all, incidents can be made up of multiple attacks. But it is more complicated than that. These multiple attacks could not only be classified differently, but could also involve multiple attackers who are attacking multiple targets. As has been stated previously, what distinguishes one incident from another is the distinctiveness of the attackers, and the degree of similarity of sites, techniques, and timing. It does not mean that attackers, sites, techniques and timing are identical. As such, the determination of the scope and characteristics of an incident, and then its classification must be accomplished in an atmosphere of uncertainty. Nevertheless, as has been discussed in this dissertation, it is important to do so. Indeed, this is routinely done by CERT®/CC personnel, and it was done for this research. However, for both the CERT®/CC and myself, this process was informal and uncertain, particularly with respect to determining the scope of the incidents. For CERT®/CC personnel, this involved meetings where information was exchanged and then correlated. In the early years of the CERT®/CC, these were often chance encounters "around the coffee pot." In more recent years, information exchange took place in periodic meetings. For this research, this judgment of CERT®/CC personnel was combined with comparisons between the records to determine the scope of each incident. This ad hoc process will not scale up as the Internet grows exponentially. A more formal process is required. In addition, unless more personnel can be assigned to incidents response, automated software tools will be necessary for the more routine incidents, leaving personnel free to determine the scope of only those incidents with the greatest uncertainty. The following sections will discuss first, how incidents classification was done at the CERT®/CC, second, how it was done for this research, and third, how incident classification should be accomplished as a result of this research. The three steps for full classification of an incident are 1) determine the scope, 2) determine the characteristics, and 3) determine the classifications. Development of a more formal process to determine the scope of incidents, and software tools to automate part of that process are discussed in Chapter 15. 13.3.1 Classifications at the CERT®/CC during the period of research - The process of classification of incidents at the CERT®/CC during the period of this research was described briefly in Chapter 4. As noted there, after December, 1993, the process included summaries, and after the summer of 1994, the summary records were relatively complete records of the incidents. These summary files represented classifications of the incidents. As Chapter 4 indicates, these summaries contained the following: 1. A file identification consisting of the key letters CERT®#, INFO#, or VUL# followed by a randomly generated, but unique, number, 2. Reporting date, 3. Notes and excerpts from e-mail and other files sent to the CERT®/CC, 4. An identification number for each excerpt that could be used to retrieve the original file, 5. List of sites involved, 6. Lists of keywords describing attacker activity in various categories. The last item, the lists of keywords, was related to the classifications of the taxonomy for this research in Chapters 7 and 8. CERT®/CC action was initiated when the incident activity, information request, or vulnerability information was reported. If it was deemed appropriate, a CERT®, INFO, or VUL number was assigned, which then automatically resulted in a summary file being opened. As stated in Chapter 4, the correspondence between incidents and summaries was not one-to-one. Some of the summaries initially opened by the CERT®/CC later proved to be related to each other. Once CERT®/CC personnel determined that two or more summaries were related, the usual course of action was to indicate this relationship in the summaries, but to keep all the summaries open. As such, the number of summaries in the CERT®/CC records was greater than the number of actual incidents. Occasionally, a summary was closed and the information from that summary was copied to a related summary. CERT®/CC personnel did not classify the attacks within each incident. Instead, they recorded keywords as described in Chapters 4, 7 and 8. This process was not consistent for several reasons. First, different terms were used in different incidents to describe the same type of attack. Second, while most of the summaries contained at least one key word that described the level of access obtained by the attacker, many of the summaries contained few, if any, other keywords. This meant the CERT®/CC summaries often did not contain complete details of the incidents. 13.3.2. Classification of Incidents for this Research - In order to gather data about incidents during the period of this research, the incidents had to be created from the CERT®/CC summaries. As described in Chapter 4, this was a difficult and time-consuming process, particularly since this was done after the incidents were closed. As stated in that chapter, the summary records were searched first by reading the summary, and then Unix search tools, such as the grep utility, were used to relate key words and phrases to the incidents already created. The four types of key words and phrases were: 1) Notes by CERT®/CC personnel indicating a relationship with other summaries, 2) Site names, 3) Keywords from the categories of the taxonomy (tools, vulnerabilities, access level, etc.), 4) Unique words, phrases or letters. As stated in Chapter 4, the first of these categories, the judgment of CERT®/CC personnel, was given strong weight in determining the scope of an incident. Experience with this research found that, when searching with the Unix utilities, site names was the best category of words or phrases to use. However, unique words or phrases were a particularly good way to reduce uncertainty. An example of this was given in Chapter 10 where the "Dutch hacker" incident was described (Section 10.2.1). In this incident, the intruders often installed a backdoor process operating on socket 87. Therefore, the keywords "socket 87" or "87 socket" were definite indications of a relationship when they were found in the CERT®/CC records. Once the incident records were reconstructed, the keywords in the records were used to determine the following: 1) an overall classification of the incident according to either the highest level of access the intruder obtained (root, account, or attempt), or the type of unauthorized use (denial-of-service, disclosure of information, or corruption of information) 2) the presence of keywords from the other categories of the taxonomy, including attackers, tools, vulnerabilities, results and objectives There were several difficulties with this approach. First, as discussed in Chapter 8, numerous categories in the taxonomy (Figure 6.9) had little information in the CERT®/CC records, and, as noted above, the use of various terms was inconsistent. Second, inaccuracies were introduced because the classification was done by someone who had not participated in the response to the incident (me) after the incident was closed . Finally, this process was very labor intensive, making it essentially unrepeatable. 13.3.3. Recommended Process for Classifying Incidents - The following sections outline a recommendation for a process to classifying incidents, based on experience with this research. 13.3.3.1. Determining Incident Scope - In the short term, the process of determining the scope of an incident could be improved by taking two steps. First, only one incident summary should be maintained for each incident, open or closed. When a relationship is found between two summaries indicating they are part of one incident, they should be combined under one summary. Unfortunately, it would be difficult to discontinue the use of a particular CERT® number, because CERT® numbers are used in communications with the affected sites. One possible approach would be to have each incident summary identified with an incident number for internal CERT®/CC use only. The CERT® numbers would continue to be used in communications with affected sites, but related records would be stored within one summary file under the separate incident number. 13.3.3.2. Determining Incident Characteristics - The process of searching records for keywords in order to determine the scope of an incident will work more effectively if a standard set of keywords and phrases is recorded in each summary. Recording these incident characteristics during an incident will make the information more timely and accurate. Experience during this research suggests several fields of data are useful to record. These fields are grouped in the following categories: time and duration, sites, workload, attacks, and responses. A) Time and Duration: 1) Reporting Date - the date the incident was reported to the CERT®/CC 2) Starting Date - the earliest date of known intruder activity 3) Ending Date - the latest date of known intruder activity (note, this is not the date the incident was closed, which is an administrative action unrelated to intruder activity) B) Sites: 4) Number of Reporting Sites - the total number of sites reporting the incident 5) Reporting Sites - the site names of each site that reported the incident 6) Number of Other Sites - the total number of sites involved, but not reporting the incident 7) Other Sites - the site names of other sites involved, but not reporting the incident C) Response Workload: 8) Number of Messages - the number of messages to/from the CERT®/CC (or some other appropriate measures of CERT®/CC workload with respect to the incident) D) Attack Activity: 9) Attackers - keywords identifying attackers and their categories (from Figure 6.9) 10) Tools - keywords identifying tools and their categories (from Figure 6.9) 11) Vulnerabilities - keywords identifying vulnerabilities and their categories (from Figure 6.9) 12) Level - keywords identifying the types of unauthorized access or unauthorized use, and the highest level achieved by the attackers (from Figure 6.9) 13) Results - keywords identifying results and their categories (from Figure 6.9) 14) Objectives - keywords identifying objectives and their categories (from Figure 6.9) E) Response to Attacks: 15) Corrective Actions - keywords identifying corrective actions taken at the sites involved, which could be categorized as internal actions (restrict hardware/software, configure hardware/software, upgrade system, or preventive measures), external actions (actions against intruders, or law enforcement), or other appropriate categories Experience with recording of these data will probably show that these categories should be modified or additional categories should be added. It is important, however, for the data that is recorded and the keywords that are used, to be defined, systematic and consistent. New keywords should only be added when intruder activity cannot be described by the existing set of keywords, and only when accompanied by a detailed description of the keyword. 13.3.3.3. Classification of Incidents - Implicitly, CERT®/CC assigned an overall classification to each incident according to the highest level of access obtained by any intruder (root, account or access) for the unauthorized access incidents (90% of all incidents). For unauthorized use incidents, general or average intruder activity was used to determine a category for the incident (denial-of-service, corruption of information, disclosure of information). This process was also followed for this research and was found to be a satisfactory overall classification for an incident. It is recommended this process be continued. Such a classification was useful in determining overall activity. The frequency of occurrence of activity in the various categories of the taxonomy could be determined from the keywords describing the incident. This was done for this research (Chapter 8), and should be improved as these data are recorded more systematically. 13.4. Summary of the Utility of the Taxonomy of Computer and Network Attacks The taxonomy developed for the classification of attacks was found to be satisfactory for classifying the CERT®/CC records. It should be expected, however, that a satisfactory taxonomy would be limited in some of the desired characteristics. This was found in this research also. In general, in applying the taxonomy for this research to the CERT®/CC incidents, there were few instances when a single classification was not directly determined; an indication that taxonomy categories were mutually exclusive. Two problems were noted. First, when there was limited information, there was sometimes difficulty in assigning an attack to a single category. Second, one attack could sometimes, theoretically, be in two categories. During the classification of the data for this research, there was no instance when a category could not be found for some data, which was an indication that the taxonomy categories were exhaustive. Two adjustments to the taxonomy were made during the research to ensure that the categories were theoretically exhaustive. First, the data in transit category was added to the access block. Second, the distributed tool and data tap categories were added to the tools block. The criteria that the categories to be unambiguous, that the classifications to be repeatable, and that the terms to be intuitive and acceptable could not be tested with the data used in this research, because classifications were made once by one person. Additional testing by the CERT®/CC, other response teams, and other researchers, could be an opportunity for such an evaluation. The conclusions from this research were largely drawn from analyzing data that had been classified using the taxonomy of attacks. The analysis could not have been conducted without such a taxonomy for classification. The taxonomy is also potentially useful because it can organize thinking about computer and network security to emphasize that, in order to be successful, an attacker must find one or more paths that connect the attackers to their objective. Computer security efforts can be aimed at the six blocks of the taxonomy. The process of determining the scope of an incident could be improved by maintaining only one internal incident summary for each incident, open or closed, and by using a formal process of searching records for keywords and phrases, in addition to other methods to collapse attacks into incidents. A standard set of keywords and phrases that are defined, systematic and consistent, should be recorded in each summary. New keywords should only be added when intruder activity cannot be described by the existing set of keywords, and only when accompanied by a detailed description of the keyword. Suggested fields of information to record are reporting date, starting date, ending date, number of reporting sites, reporting sites, number of other sites, other sites, number of messages, attackers, tools, vulnerabilities, level, results, objectives, and corrective actions. Experience with recording of these data will probably show that these categories should be modified or additional categories should be added. It is important, however, for the data that is recorded and the keywords that are used, to be defined, systematic and consistent.
An overall classification should be given to each incident according
to the worst level of unauthorized access or unauthorized use.
The frequency of occurrence of activity in the various categories
of the taxonomy could be determined from the keywords describing
the incident.  [13]
 |
