Welcome back from the Thanksgiving holiday break. The good news is that this lecture will come to you about Christmas, therefore, no homework. The not so good news is that this concluding Lecture 4 on Substitution with Variants covers some difficult material of wide practically in the field.

In Lecture 4, we complete our look into English monoalphabetic substitution ciphers, by describing multiliteral substitution with difficult variants. The Homophonic and GrandPre Ciphers will be covered. The use of isologs is demonstrated. A synoptic diagram of the substitution ciphers described in Lectures 1-4 will be presented.

Each English letter in plain text has a characteristic frequency which affords definite clues in the solution of simple monoalphabetic ciphers. Associations which individual letters form in combining to make up words, and the peculiarities which certain of them manifest in plain text, afford further direct clues by means of which ordinary monoalphabetic substitution encipherments of such plain text may be readily solved. [FR1]

Cryptographers have devised methods for disguising, suppressing, or eliminating the foregoing characteristics in the cryptograms produced by methods described in Lectures 1-3. One category of methods called "variants or variant values" is that in which the letters of the plain component of a cipher alphabet are assigned two or more cipher equivalents.

Systems involving variants are generally multiliteral. In such systems, there are a large number of equivalents made available by combinations and permutations of a limited number of elements, each letter of the plain text may be represented by several multiliteral cipher equivalents which may be selected at random. For example, if 3-letter combinations are employed as multiliteral equivalents, there are 26³ or 17,576 available equivalents for the 26 letters of the plain text.

They may be assigned in equal numbers of different equivalents for the 26 letters, in which case each letter would be representable by 676 different 3 letter equivalents or they be assigned on some other basis, for example proportionately to the relative frequencies of the plain text letters. [FR1]

The primary object of substitution with variants is again to provide several values which may be employed at random in a simple substitution of cipher equivalents for the plain text letters.

As a slight diversion, the reader may ask about uniliteral substitution with variants. It is but not very practical. Note the following cipher alphabet constructed in French by Captain Roger Baudouin in reference [BAUD]:

(Note that the Captain was not an ACA member. The H=H combination is not allowed.)

Baudouin proposed that the J and Y plain be replaced by I plain and K plain by C plain or Q plain and W plain by VV plain. Four cipher letters would be available as variants for the high- frequency plain text letters in French.

Mixed alphabets formed by including all repeated letters of the key word or key phrase in the cipher component were common in Edgar Allen Poe's day but are impractical because they are ambiguous, making decipherment difficult; for example:

Enciphering Alphabet:

Inverse form for deciphering:

The average cipher clerk would have difficulty in decrypting a cipher group such as TOOET, each letter having 3 or more equivalents, from which plain text fragments (n)inth, ft thi(s), it thi, etc. can be formed on decipherment. [FR1]

In simple or single-equivalent monoalphabetic substitution with variants, two points are evident:
 

1) the same letter of the plain text is invariably represented by but one and always the same character or cipher unit of the cryptogram.

2) The same character or cipher unit of the cryptogram invariably represents one and always the same letter of the plain text.

 

1) the same letter of the plain text may be represented by one or more different characters or cipher units of the cryptogram. But,

2) The same character or cipher unit of the cryptogram nevertheless invariably represents one and always the same letter of the plain text.

 

Figure 4-1
 

Figure 4-2
 

Figure 4-3
 

Figure 4-5
 

Figure 4-6
 

Figure 4-7
 

Figure 4-8
 

Figure 4-9
 

Figure 4-10
 

The matrices in Figures 4 -1 to 4-10 represent some of the simpler means for accomplishing monoalphabetic substitution with variants. The matrices are extensions of the basic ideas of multiliteral substitution presented in Lecture 3.

The variant equivalents for any plain text letter may be chosen at will; thus, in Figure 4-1, e= 10, 15, 60, or 65; in Figure 4-2, e= AU, AZ, FU, FZ, LU or LZ.

Encipherment by means of matrices shown in Figures 4-2, 4-3, 4-6 is commutative. The coordinates may be read row by column or visa versa. There is no cryptographic ambiguity. The remaining matrices are noncommutative. The general convention is to read row by column.

In Figures 4-5 and 4-6, the letters in the square have been inscribed in such a manner that, coupled with the particular arrangement of the row and column coordinates, the number of variants available for each plain text letter is roughly proportional to the frequencies of the letters in the plain text. Figure 35 incorporates a keyword on top of this idea. [FR1]

The Homophonic Cipher is a simple variant system. It is a 4-level (alphabets) dinome cipher. Consider Figure 4-11.
 

Figure 4-11
 

The keyword TRIP is found by inspecting dinomes 01, 26, 51, and 76. (The lowest number in each of the four sequences.) [FR1] [FR5]

The Russians added an interesting gimmick called the Disruption Area. Consider Figure 4-12 and note the slashes under U - X for the fourth level of dinomes. The famous VIC cipher used this feature very effectively. [NIC4]
Figure 4-12
 

The keyword NAVY is represented by dinomes 01, 27, 53, and 79.

Security for Homophonic systems is greatly improved if the dinomes and the four sequences are assigned randomly. However, the easy mnemonic feature of the keyworded four sequences is lost.

The Mexican Cipher device is a Homophonic consisting of five concentric disks, the outer disk bearing 26 letters and the other four bearing sequences 01-26, 27-52, 53-78, 79-00. The cipher disk enhances frequent key changes. Figure 4-12 shows the matrix without the disruption area. [FR5] [NIC4]

Lets solve the following cryptogram.
 

									
68321   09022   48057   65111   88648   42036   45235   09144
05764   22684   00225   57003   97357   14074   82524   40768
51058   93074   92188   47264   09328   04255   06186   79882
85144   45886   32574   55136   56019   45722   76844   68350
45219   71649   90528   65106   11886   44044   89669   70553
18491   06985   48579   33684   50957   70612   09795   29148
56109   08546   62062   65509   32800   32568   97216   44282
34031   84989   68564   53789   12530   77401   68494   38544
11368   87616   56905   20710   58864   67472   22490   09136
62851   24551   35180   14230   50886   44084   06231   12876
05579   58980   29503   99713   32720   36433   82689   04516
52263   21175   06445   72255   68951   86957   76095   67215
53049   08567   9730

If the cryptogram contains an even number of digits, as for example 494 in the previous message, this leaves open the possibility that the message is a cipher containing 247 pairs of digits; were the number of digits an exact odd multiple of five, such as 125, 135, etc., the possibility that the cryptogram is in code of the 5-figure group type must be considered.

We next study the message repetitions and what their characteristics are. If the cipher text is of 5-figure code type, then such repetitions as appear should generally be in whole groups of five digits, and they should be visible in the text just as the message stands, unless the code message has been superenciphered. If the cryptogram is a cipher, then repetitions should extend beyond the 5-digit groupings; if they conform to any definite at all they should for the most part contain even numbers of digits since each letter is probably represented by a pair (dinome) of digits.

We start with 4-part frequency distribution. We next assume a 25 character alphabet from 01-00. This is the common scheme of drawing up the alphabets. Breaking the text into dinomes (2-digit) pairs yields:
 

What we have before us are four simple, monoalphabetic frequency distributions similar to those involved in a monoalphabetic substitution cipher using standard cipher alphabets. The next step is to fit the distribution to the normal. Since I=J for the 25 letter alphabet, we find that the Keyword is JUNE and the following alphabets result:
 

If a 26-element alphabet were used only the distribution analysis would have been changed to be on a basis of 26, the process of fitting the distribution to the normal would be the same.

Suppose we know that two correspondents have been using the same variant system as in the previous Homophonic. The message intercepted is:
 

								


48226   88423   52099   93604   76059   05651   36683   52267
97114   54466   76

Levels:
 

								
(1)  22 23 09 04 05 22 11
(2)  48 36 36 35 45 44
(3)  68 52 56 51 68 67 66
(4)  84 99 76 90 97 76

So:
Levels:
 

								
(1)  22=W; 23=X; 09=I; 04=D; 05=E; 22=W; 11=L
(2)  48=X; 36=L; 36=L; 35=K; 45=U; 44=T
(3)  68=S; 52=B; 56=F; 51=A; 68=S; 67=R; 66=Q
(4)  84=I; 99=Y; 76=A; 90=P; 97=W; 76=A

The plain-component sequence is completed on the letters of the four levels by Caesar Rundown, as follows:
 

The generatrices with the best assortment of high frequency letters for the four levels are:
 

								

Level 1           	 Level 2          	Level 3          	Level 4

EFRMNET           REEDON           EOSNEDC          NCETAE

The plain text reads "Reinforcements needed a[t once]".
Looking at the equivalents 01,26, 51, 76 we reveal the keyword JUNE.

In evaluating generatrices, the sum of the arithmetic frequencies of the letters in each row may be used as an indication of the relative "goodness". A statistically better procedure uses the logarithm of the probabilities of the plain text letters forming the generatrices. See [FR2]

The Homophonic is a popular cipher and has been discussed in several issues of The Cryptogram as well as LEDGES' NOVICE NOTES. See references [HOM1 -HOM6] and [LEDG].

For our computer bugs, TATTERS Homophonic solver is very easy to use and available on the Crypto Drop Box.

Consider the cipher matrices shown in figures 4-11 to 4-13. These are called frequential matrices, since the number of cipher values available for any given plain text letter closely approximates its relative plain text frequency.

Figure 4-11
 

( 676 - cell matrix )

In figure 4-11, the number of occurrences of a particular letter within the matrix is proportional to the frequency in plain text; the letters are inscribed in random manner, in order to enhance the security of the system.

Figure 4-12
 

In figure 4-12, the same idea as 4-11 is presented in reduced form from 26 x 26 to 10 x 10. The letters have been inscribed by a simple diagonal route, from left to right, within the square, and the coordinates scrambled by means of a key word or key number.

Figure 4-13
"Grandpre"
 

Figure 4-13 illustrates the famous Grandpre Cipher; in this square ten words are inscribed containing all the letters of the alphabet and linked by a column keyword ("equivalent") as a mnemonic for inscription of the row words. ACA literature also covers this cipher. See references [LEDG] and [GRA1 - 3] for solution hints for the Grandpre cipher.

General Luigi Sacco proposed a frequential-type system that uses both enciphering and deciphering matrices. The inscribed dinomes were completely disarranged by applying a double transposition to suppress the relationships between letters. References [SACC] and [FR1] both give a good description of the process. The number of variant values in this system are reflective of the Italian language.

The Baconian ciphers found in the Cryptogram are a variant system. The "a" elements may be represented by any one of 20 consonants as variants, while the "b" elements may be represented by any one of 6 vowels; or the letters A-M may be used to represent the "a" elements and the letters N-Z for the "b" elements; digits may be used for either the "a" or "b" elements, either on the basis of first five or last five digits, or odd versus even digits, or the first 10 consonants (B-M) and the last 10 consonants (N-Z)

Friedman describes a complex variant known as the summing- trinome system. Each plain text letter is assigned a value from 1-26; this value is expressed as a trinome, the digits of which sum to the designated value of the letter. The letter assigned the value of 4 may be represented by any of 15 permutations and combinations. Friedman discusses further ways of complication including disarrangement, addition of punctuation and nulls. See [FR1] pages 109-110. Note the inverted normal distribution representation of this cipher.

The following cryptogram is available for study:
 

Note the total absence of A, E, I, O, U, and Y. Remarkable and definitely nonrandom event. Since a uniliteral substitution alphabet with 6 letters missing is highly unlikely, the next guess is we are dealing with a multiliteral substitution. Closer inspection shows that ten consonants are initials (B D G J L N Q S V X) and the remaining ten consonants are used as terminals (C F H K M P R T W Z). This implies both bipartite and biliteral character.

We construct a digraphic distribution:
 

We assume the use of a small enciphering matrix with variants for rows and columns. We assume that the various possible cipher variants are of approximately equal frequency; the column indicators pair equally often with the row indicators of the enciphering matrix. We look for similar row profiles and column profiles. We match first the rows and then the columns.

Row L and V distributions have pronounced similarities. They are "heavy" in their frequency distributions in the same places. So are rows D and N. They have homologous attributes in appearance.

Finding the next rows are not obvious. We use a "goodness of match" procedure to equate interchangeable variants. We calculate the cross-product sums for each trial. The next heavy row is G. We test G against the remaining rows.

We compare the balance of rows:
 

The results are most probably match G and S.

The next heaviest row is B. Testing against the remaining three rows we have:
 

The correct pairings are B with J and Q with X. Since we have not found more than two rows for any one set of interchangeable values the original matrix has only five rows.

Values represent the sums of the combined rows. We apply the same process to matching columns. C and H are a matched pair. F with M and P with R. We use the cross product sums for the balance of the columns.

Combinations:

We would expect that the proper pairings are K with W and T with Z.

We convert the multiliteral text to uniliteral equivalents using an arbitrary square for reduction to plain text.

The converted cryptogram is solved via the principals in Lecture 2 and Lecture 3. The beginning of the message reads Weather forecast. The original keying matrix is recovered with a keyword of ATMOSPHERIC.

The method of matching rows and columns applies equally well for all the matrices shown previously. It is key to start with the best rows and columns from not only heaviness standpoint but the distinctive crests and troughs. A second key is the low frequency letters. No variant system can adequately disguise low frequency letters and they will have the same frequency in the cipher text. Friedman describes a more general solution to variant analysis. [FRE1, p119 ff]

Chapter 10 of reference [FRE1] covers the disruption process associated with monome-dinome alphabets of Irregular-Length cipher text units. Figures 4-14 and Figure 4-15 show enciphering matrices where the encipherment is disrupted and commutative. The normal row conventions are used to encipher except when the row indicator was the same for the immediately preceding letter. In Figure 4-14, EIGHT could be encrypted 10 29 7 8 49 and then rearranged into standard groups of 5 letters (numbers). In Figure 4-15, E = 24 or 42, T = 621 or 162. Figure 4-16 is an example of the Russian disruption process added for security.

Cryptograms produced using identical plain text but subjected to different cryptographic treatment, and yielding different cipher texts are called isologs. (isos = equal and logos = word in Greek). Isologs are usually equal or nearly equal in length. Isologs, no matter how the cryptographic treatment varies, are among the most powerful tools available to the cryptanalyst to solve difficult cryptosystems.

Take two messages A and B suspected of being isologs and write them out under each other. We then examine the similarities and differences. Assume the messages both start with "Reference your message..." I will arrange the messages in a special table to facilitate the study.

								      
Group No.
			       5                          10			    15
A     82 26 56 31 03 74 83 96 98 42 32 52 97 01 15
A'    30 15 08 74 97 14 51 19 73 60 49 67 65 01 06

B     80 27 78 91 06 94 00 01 38 28 54 08 24 00 65
B'    45 64 79 91 81 69 67 25 38 89 41 56 32 52 03

C     63 62 93 39 18 43 15 88 10 48 26 45 84 50 39
C'    90 62 87 75 36 20 35 11 05 70 89 27 77 50 11

D     81 71 35 25 38 73 30 92 07 49 61 75 21 64 76
D'    35 19 99 01 38 99 97 45 02 32 04 11 58 92 16

E     38 72 89 11 47 99 92 64 14 68 13 36 53 38 81
E'    38 46 31 75 47 14 64 80 06 46 85 86 45 38 98

F     89 69 79 38 16 51 75 05 70 74 11 80 44 32 55
F'    26 12 18 38 78 94 88 93 37 28 11 27 22 05 04

G     28 12 02 77 30 31 19 97 99 62 27 86 56 06 53
G'    06 48 43 21 03 98 71 54 26 62 80 76 08 98 80

H     90 87 04 08 67 46 59 41 98 55 10 82 22 29 87
H'    44 10 55 29 00 59 72 82 28 55 87 30 07 08 93

J     46 72 93 62 45
J'    59 68 24 62 53