Advice on securing a sensitive Access database

L

Les Desser

"(e-mail address removed)" <[email protected]> Wed, 16
Apr 2008 01:24:04 writes

(As I worked through your notes, later parts answered some earlier
questions. I have removed some but may have left some others by
mistake)
What I did here was to build a function that wasa used for both
encryption and decryption of a field,

The same function to do both?
based on the AES algorithm.
Effectively you can choose how many AES keys are used for securing the
data. I used one AES key per table.
A *different* key for each table?
I applied the function to encrypt / decrypt such that any data read
from the table was unintelligible without decrypting it through the
function, and in turn no data was written to the table without being
encrypted with the function. In this way each field was encrypted with
the same AES key. This was done through unbound forms and code. The
end user never actually saw the encryption taking place.
An Access question: Could controls not be bound to the decrypt
function?
The AES key for each table was stored as a table property with DAO
code, encrypted Asymmetrically (public / private), for each user that
had access to the table. Each users public key was used to encrypt a
copy of the AES key for the table and store that as the value for the
property. Only the users private key would be able to decrypt the copy
of the AES key stored as a property specific to that user (eg/ I made
a table property with the same name as the username, and stored the
encrypted (users public key) form of the AES key as the value of that
property).
So we have each user with their own encrypted copy of the key.
I actually kept all the encryption keys stored in the back end, and
only the functionality to use them in the front end. This way the
front end never really needed changing once the encrypt / decrypt
functionality had been built in.
Understood

Everything stored in the backend db was in ciphertext (encrypted
form). This way it did not matter if someone stole a copy of the
backend db, it was effectively useless. If they connected via code
from a different application to try and read the data all they would
get is meaningless rubbish from each field.

[... balance of notes left for later digestion..]
The user access to each table was done via checking if the user had a
table property in their name, with a stored AES key value. This was
also able to be checked for validity (ie/ to see if someone had just
copied the property from another table).

The way the user asymmetric keys were used is as follows. Bare with me
it takes a little to go through it.

1/ A Private / Public key pair is generated for a user in the Crypto
Admin application.
2/ The user is (in the crypto admin app) 'assigned' the tables that
they are allowed to access
3/ For each table that the user is allowed access the users public key
is used to encrypt the appropriate tables AES key, which is then
stored as a table property using the users name as the property name.
4/ The users stored encrypted copy of a tables AES key can be checked
for validity by either using the decrypted AES key to decypher a known
value and see that it is true (such as a table property that holds a
copy in encrypted form of the tables name)

That is OK to check one user's key being copied to an other table. What
about one user's key being copied to the same table under a different
user's name?

Storing an encrypted copy of the table name and the user name together
with the key should stop that.
, or by placing a MD5 hash
value with the stored AES key that matches the users name or password
or some other known value. I went with the latter, but the former is
probably easier to do.

Actually if I were to do this again, I would make a table property and
store an MD5 hash of the tables name in it, encrypted with the tables
AES key. When a user tries to access the table the form (code) checks
to see if the user has an associated table property in their name,
then uses the users private key to decrypt the stored encrypted AES
key, then uses the AES key to decypt the stored MD5 hash (the known
value) and checks this against the MD5 hash generated at runtime for
the tables actual name. If they match then the user is valid for the
table, if the MD5 hashes dont match then something has either gone
wrong or someone has copied the username / stored value from another
table and is using it to try and break the table in question.
Don't you also need to check in the same way in case the property has
been copied on the same table for a different user?
This way, with code, you can assign different users access to
different tables without fear that because they have access to one
area of data that they could access other areas that they may not be
allowed to.
Makes a lot of sense
The Crypto Admin part was to assign these users to the tables and
associate the keys properly. It made life a lot easier than trying to
do this through the front end, and allowed the private keys and AES
keys another layer of security by never having them directly available
to the 'public'.


This is absolutely correct. I would recommend doing the encryption
control through a third application as I talked about above. Use code
for everything that needs encryption, and hard code the needed
functionality into the front end. Keep the admin separate from the
front end, and keep the data in the back end.


I hope this helps clear this up a little. As I said it was a pain to
do this. The weak point as I mentioned earlier was in the storage of
the usernames and passwords (with the private keys). I was giving this
a little thought since your first post, and *maybe* have a better way
to do it than the one I first used.
It occurs to me that it would be better to keep the usernames
completely obfuscated if possible so that it makes things very hard
for someone to be able to reverse engineer them. For this you could
use again MD5 hashing. for the users login, they would type their
username and password. Both the username and password are MD5 hashed.
The front end checks a table in the back end for a matching value for
the username. If this is found then the password MD5 hash is used as
an AES key to decrypt the users private key, and some known value
check for validity same as mentioned before.

As with all cryptographic applications, it is a complex task to get it
right.

Say that again!
Even the best cryptographic ciphers can be undone by poor
system design (think Enigma in WW2). In this case the weakest point as
I see it is the username / password area used for the login to get the
users private key. If you are able to overcome that with a better
system design then go for it.

I wonder if some hardware would help. Fingerprint reader? (I have no
idea how secure they are)
I would recommend it if the data is
truly valuable. The best you could realistically go for here is tri-
factor security, something you have (a token), something you know
(username / password), and something you are (biometric). Might be
overkill, but keeping the users private key out of the system would
make this application really strong.

Don't understand "keeping the users private key out of the system"
If you can get to the dual factor
level that would be brilliant for most purposes.

Hope this helps

Very much so! I am at the stage that as I work through your notes I
think I understand each step but I cannot say I have a clear picture in
my head of all the steps. I need to re-read a few more times.
 
T

The Frog

Hi Les,

We have a lot of points here to cover, so bare with me as I attempt to
work through them for you one by one:........
The AES key for each table was stored as a table property with DAO
code, encrypted Asymmetrically (public / private), for each user that
had access to the table.


Why a table property rather than as a separate table? Just to make
it
not so obvious?


***Answer: The reason for this was more of a matter of design
philosophy. You could also do this as a table. I preferred to make the
system as difficult to duplicate as possible. That being said an
experienced Access Programmer would still be able to locate the extra
table properties. It just helped to rule out the 'middle' level users
who know enough to be a bother but not enough to be a serious threat.
The security of the data ultimately does not depend on the obfuscation
of the key storage, but rather on the strength of the ciphers used.

What I did here was to build a function that wasa used for both
encryption and decryption of a field,

The same function to do both?


*******Answer: With symmetric encryption the same function is used for
both encryption and decryption purposes. You really only need one
function here, but if you wish to make the code more understandable
you could always create two functions, or use an object (as done in
one of the implementations that yields a class module) so that you
have three properties of the class, one for the AES key, and one for
each the encrypted and decrypted data (plaintext and ciphertext). You
could build in functionality so that if the value stored in the
encrypted property was changed by placing a new value from the app to
the object that the decrypted would automatically reflect the change.
I chose to ditch the class module in favour of a function (If I
remember correctly......).

Effectively you can choose how many AES keys are used for securing the
data. I used one AES key per table.


A *different* key for each table?

****Answer: Yep, exactly that, a different key for each table :)


function?

****Answer: It depends on your design philosophy. I chose to use code
to control every aspect of each form and report. This made for a
little more work in the construction of the forms, particularly with
regards to list boxes and the like, but in the end the functions
needed to populate the controls with appropriately decrypted data only
had to be written once, and then called from the form (again by code)
to actually fill the control with the decrypted data. There are a
myriad of ways you could approach this, so it really is just a matter
of working out one that suits the design of your app and reverse
engineering it into the existing forms, reports, etc...

about one user's key being copied to the same table under a different
user's name?

Storing an encrypted copy of the table name and the user name
together
with the key should stop that.

Don't you also need to check in the same way in case the property has
been copied on the same table for a different user?


****Answer: Because of the private key / public key way of enciphering
the tables AES key, only the correct private key will decrypt a users
enciphered AES table key. This means that if a user duplicates another
users table key and renames it for themselves (lets say), then it
still wont help them unless they possess the specific private key for
the table key they copied. Their own private key is unique to them and
so wont work with anyone elses public key encrypted data. It only
works for data that is (in this case the table key) that is encrypted
with their matching public key. The key pairs are unique, and as such
the protection of the private key is really important, hence also my
comments on keeping the private keys out of the system if possible.

idea how secure they are)


****Answer: With regards to using hardware for storing the private
keys and also for biometric authentication you need to look around at
this stuff. There are a lot fo rubbish components on the market and
very few that are actually reliable AND secure. One of the best that I
have seen for securely storing keys was from Rainbow Technologies
(this is not a product endorsement, but rather an endorsement of the
type of approach used by this technology) - they had / have a product
called the iKey that acted as a storage container, a very secure one,
for keys / certificates, and if I remember correctly they also had a
model that could do cryptographic calculations. It was basically a USB
key, so no special hardware was needed on the computer with the
application, just some drivers. I saw this about ten years ago now, so
I would expect that today there are many versions of this sort of
thing available. If you can find one that has a simple to administer
system for creating / removing users and keys, and can be integrated
into your app without too much hassle then I would certainly entertain
the idea. It was what I wanted to do with the app I built. I was
hoping to find a USB key, with the ability to store the Public key for
a user as well as have it password protected on the key, and with an
integrated fingerprint/thimbprint reader. This would have been a nice
tri-factor authentication system. Cost is also a factor here too, some
of these things can be pretty expensive from memory.

I have also seen keyboards that have integrated smart card readers
(credit card tpe cards with smart chips on them) and finger /
thumbprint readers. I have also heard of software that can tell who
you are by just the way you type on the keyboard. Voice is another
possibility, but of course it can be recorded.

A passing thought on security here too. If your data is **REALLY**
sensitive, and the possibility exists that a user may be co-erced by
force into accessing the system / information, you may want to think
about placing a dummy table with false data that LOOKS real enough.
What you do is to have the user enter their password backwards or
something like that when under duress, and on login check it, and if
the password is entered backwards then only show the rubbish fake data
- but make the application look like it is working perfectly. At the
same time send a message to someone to let them know of the intrusion
and duress situation. Pretty extreme and certainly not an everyday
thing, but I have seen situations where this is necessary.


****Answer: I think we covered this above, but again briefly it comes
down to a matter of adequately securing the system. The private /
public key pairs are the core of getting access to the tables AES
keys. What we need to do to really make it safe is to keep the private
key as secure as possible. Because the private keys, in the model I
ended up deploying, are stored in the database itself (although
encrypted), they represent a risk to the security of the system. Even
though the encryption is strong (AES 256), the password that the user
chooses becomes the weak point - it could be guessed or forced from
the user - and hence the private key becomes available - hence the
data becomes available. By keeping the private key separate
completely, it does not matter if the user password is guessable or
not - you are just eliminating risk from the system design.

If you use longer passwords as a minimum then users tend to write them
down or use easily rememberable phrases that can be guessed. This goes
back to the point about tri-factor authentication. That is considered
by many to be very strong, but even dual factor would provide a
massive increase in the level of security for the application - in
this case username/password (something you know) and a separate device/
storage for the private key (something you have). The 'something you
are' part would put the icing on the cake so to speak.

So in short, storing the private keys is the weakest point in my
existing app, due to the fact that a users password may be obtainable.
Keeping the private keys out of the system goes a long way to
eliminating this deficiency, and having a biometric pretty well
completes it. Another possibility would be to require two users to
authenticate themselves before the system was functional, and there
are cryptographic methods that can achieve this. Its all a matter of
approach. How far do you want to go to protect the data? What is the
cost of having unauthorised access? Basically you need a risk analysis
to figure out how far to go. There is always another level of security
you can add, the trick is knowing how little is too little and how
much is too much.


available. (from the third posting)

****Answer: That is exactly correct, the private key for the user
needs to be made available to the user when they perform a successful
login. Only then should the users private key be available to them and
the application. The public key can be seen by anyone so it doesnt
really matter.

The users private key is NEEDED to obtain and decrypt the tables AES
key. Thats why we have a copy of the AES key encrypted with table for
each user, using each users unique public key.

I think what you missed here was that each user gets a unique public /
private assymmetric key pair. Only the AES key is common, and only
then on a per table basis. Its kind of like having a lock box for each
user, and each lock box has a unique key that only that user has.
Inside each lock box is another key, lets say to the beer fridge :).
If I take my lock box, using my key and open it, then I can get access
to the beer fridge key and hence the beer. If however I take my key,
and I try and use it to open another users lock box it wont work
because it needs that users key and not mine - so I cant get the berr
fridge key and hence no beer :-(

What we have in this design is the same beer fridge thinking for each
table, in effect a series of independantly locked beer fridges - the
key from one beer fridge wont open another fridge. We have for each
fridge a set of lock boxes, one for each user, each secured with the
users lock (same lock for one user across all of their lock boxes).
The user can take their key, open their lock box if they have one for
a specific fridge, then take the key to the fridge and get some beer.
The user cant open another users lock box, and they cant take a key
from one fridge and use it in another fridge. BUT, because the user
has only a single key for all of their lock boxes (and hence all the
fridges that they can get beer from because they can get the fridge
key), it is very important to protect the users private key. That
private key in this analogy is the unique user private key they obtain
when they log in, and in my app is stored in the back end database -
and hence also my strong desire to keep the key away from the database
and potentially weak passwords.

I hope this helps a bit. I know this can be a tricky area to deal
with. As I said before, the application of the cryptography is the key
to success here. Its worth taking a little extra time to get the
details worked out for the implementation. The algorithms are actually
only useful if they are applied in a secure system design. The most
common cryptographic mistake I have seen is people using really
capable algorithms and really poor system design - effectivley putting
a steel door on the front of the home and a fly wire screen on the
back. If you can get the private keys out of the application /
database and store them separately and securely then do it!

Cheers

The Frog
 
L

Les Desser

Needless to say this was a time consuming process but a nice safety
feature to have.

I know this is really an "impossible" question to answer accurately, but
....

What sort of effort - in man days - should this project take in Access
for an experienced Access developer (ignoring research on the cryptology
side).
 
L

Les Desser

<ab1480ca-ca08-4af9-bf34-2840a3ea939c@a70g2000hsh.googlegroups.com>, The

[...]
I hope this helps a bit.

More than a bit - I think I now do understand the issues and techniques
involved.

I am very grateful for you long and comprehensive reply. I am now going
off to have a quick read of Cryptography for Dummies.

I'm sure I'll be back :)

Many thanks again.
 
M

Mr.Frog.to.you

Anytime. Come back with any questions you may have and I will do my
best to answer them. Another good read for a budding cryptologist is a
book by Simon Singh called 'The Code Book'. It gives you an
understanding of what the different types of cryptography are as well
as how they have and are used. Looking back through history with this
book shows quite clearly where systems have failed not because the
cryptographic standard was insufficient, but rather that the
implementation was insuffucient and made an otherwise strong cipher
vulnerable.

Its all in the way you do it :)

Cheers

The Frog
 
L

Les Desser

The AES key for each table was stored as a table property with DAO
code, encrypted Asymmetrically (public / private), for each user that
had access to the table.

I have been going carefully through your posts and am progressing well.

Just one query at this point. Why cannot the AES key be encrypted with
a second symmetric key rather than Asymmetric?

In any event, you are not publishing the public key and the private key
has to be kept secret so why not just use AES a second time?
 
T

The Frog

Hi Les,

The use of the Public / Private key pairs is simply for the separation
of users from one another. By using the key pairs for each user you
have the ability to grant / revoke permissions on a table by table
basis, or alternatively remove a user from access completely without
interrupting any other users of the system.

In essence, if you were to use a single AES key as the access code for
all users, say for example they type in a username / password and get
the AES key back in return, you have the problem of how to secure that
AES key. Where would you store it? By using the public / private key
pairs you dont have to worry about that problem because the only
exposed information is able to be publicly exposed. In this way you
can store any AES key or keys or data with a public / private key
pair.

The question then becomes why not use the public / private key pairs
as the primary means of encrypting all the data. There are two reasons
for this, the first being speed. Public / Private key encryption is
much slower to operate than symmetric encryption such as AES or DES
etc... It is theoretically possible to operate this way, but the
'horsepower' needed to do it in a timely manner for any significant
amount of data becomes a serious problem, even with current hardware.

The second reason is that you need to keep a 'central' copy of the
data available to be worked with for each user. If you use Public /
Private to encrypt this data you would require that each user has a
unique copy of the data as only their key pair could properly would
with that encrypted copy.

In short then, the basis for using the two schemes together is simply
one of practicality. We use the public / private key pairs to secure a
unique (user specific) copy of the 'central' AES key(s) that are
needed to get at the data. Each user logs in, their private key
generated from their password and username, decrypts the AES keys as
needed to access the data they are allowed to access. When you need to
remove a user from the system you can remain confident that that the
data will remain secure and simply remove the users username /
password from the system and not disrupt any other user in the system.

In theory you could also use AES or another symmetric cipher as the
means of securing the other AES keys. The condition being that you
still maintain the separation between the users username / password
key, and the key(s) needed to get at the data. If you do this it can
work fine, but you will lose the ability to operate with things like
secure tokens and such which typically work with certificates and
asymmetric encryption schemes. If you dont need / want the option to
employ tri-factor security then you can go with the more simple
username and password only. You would need to make some sort of check
in the authentication system / login so that the app knows if the user
has entered the correct data of course, same as before, then just make
sure that a copy of the tables AES key is encrypted for each user with
the users AES key. I think this should pretty much work the same for
you if you dont need the extra authentication of a smart card / token.

Is this what you were asking? I hope I understood you correctly. If
you need more info just let me know :)

Cheers

The Frog
 
L

Les Desser

We have progressed a long way and we are testing the encryption
functions and have seen some odd behaviour and wonder if there is a
problem.

We have a query that shows the original text, the encrypted string and
then the decrypted string - it is working fine.

However, there is a certain pattern of results that we do not understand
where we have identical string values in different records (user name
and event log text).

In the case of user name (8 chars), we have records with identical
strings and the encrypted values (32 chars) are totally different each
time.

So question one is if the initial values are the same why are the
encrypted values not the same (we are not using an initialization
vector)

In the case of the log text (20 chars) the first 32 characters of the
encrypted string is always the same while the remaining 32 are always
different.

So question two is why are we seeing this odd behaviour.

We intend using an initialisation vector to stop this happening but
would still like to understand what is going on.
 
T

The Frog

Hi Les,

I just want to make sure I understand this correctly so that I can
answer properly. What I think you are saying is that when using the
same (I am assuming AES) key to encrypt some text / data, that you are
seeing different outputs in different parts of the program.

If this is the case I suspect that the source of this lies in the
string / plaintext to byte conversion (and possibly the reverse also).
Theoretically the operation to encrypt (without padding) should
produce the same result for the same text / data each time it is done
if using the same key.

Things to check for are these:
1/ Are the keys being used actually the same
2/ Is the data decrypting properly
3/ If you are operating the code "By Hand" in the editor and encrypt
something does the result match the same when using the application
4/ Are the sections of code that are used to encrypt and decrypt the
same? (ie/ is the process and encryption / decryption methods being
used the same)
5/ Does the data decrypt properly?
6/ Is the source (plaintext) exactly the same when doing the testing.

The most common issue I have come across was the conversion of the
plaintext to a bytearray was producing different bytearrays. As it
turnded out the code that was feeding the plaintext to the encryption
algorithm was actually producing different plaintexts that looked the
same on screen (so to speak) but were in fact different. This came in
my case from some simple data type conversions that I wasnt handling
properly. The root cause was the variant data type being used to house
the plaintext instead of a string. When dealing with numbers in
particular it caused issues. Using strict data types eliminated the
problem.

Aside from that, the only other thing that I could think of is that
there may be an issue with the algorithm implementation itself. If you
like we can try and beat the source code to death to see if there are
any uncooperative lines of code :) Please understand I am not asking
to post your proprietary code here or data, just the algorithm
implementation code so that if you think we need to dissect it we can.

Please keep me posted, and try the "By Hand" testing with the same
data multiple times to see if you get the same strange behaviour.

Cheers

The Frog
 
L

Les Desser

<4ee80c7b-d413-470d-bb04-eae559e028c5@z72g2000hsb.googlegroups.com>, The

[...]
Please keep me posted,

Thank you for your previous post. We have introduced an IV so the
problem has gone away. I still want to understand what was going on so
will do some more test when I can.

One side affect of the encryption has surfaced. Obvious really, but did
not think of it.

If fields are encrypted then basically all indexes go out of the window.

So to fill a pull-down list from a table, the whole table needs to be
decrypted (we are using a query calling the decrypt function) and then
Access has to build a temporary index to sort.

For 26K records the decrypt is instant (query without sort) but with the
sort it is about 16 seconds.

Two solutions we can think of. Have a separate field containing the
first character in clear text or equivalent ASCI code. Or introduce our
own sequence field. Both solutions require these fields to be
maintained every time the text is changed or a record created.
 
T

The Frog

Hi Les,

Yes, indexing can be slightly problematic. One thing I did to help was
to populate a recordset with the results of the 'decryption query',
and then did my sorting / filtering from there. It is much faster to
operate in memory than off the HD. I dont know if this will suit your
situation though. I found working with ADO disconnected recordsets
quite efficient for this purpose, and you can keep the data secure
even across the 'wire' this way by having a two stage process:
1/ first query simply gets the data 'raw' from the encrypted recordset
still in encrypted form
2/ Make a function that takes the 'raw' recordset and returns a
decrypted one.

Let me know if you want to bounce this around some more.

Cheers

The Frog
 
L

Les Desser

The said:
Let me know if you want to bounce this around some more.

I would just like to tender my sincere thanks for all your help.

The project is coming to an end - based primarily on your advice and
help.

Many thanks once again.
 
T

The Frog

Hi Les,

You are most welcome. I am glad that I have had the opportunity to
help you out and hopefully add a small contribution to this group as
well. I have enjoyed helping with this. I also wish you all success
with the project. Drop the group a message sometime and let us know
how it has all worked out. My guess is that with the standards
employed here the security on the DB should be pretty damn solid!

Congratulations, Well done, and thankyou too for your feedback - I
really appreciate it :)

Cheers

The Frog
 
L

Les Desser

The said:
Drop the group a message sometime and let us know how it has all worked
out.

Once it goes live I will try and remember to let you know how it went.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top