The Joys of Importing Data

In high school I had the worst job ever! A data entry job for some no-name company. It didn't take long for me to realize that I needed good grades because I didn't want to be doing data entry for the rest of my life.

When shopping for an online SaaS tool the important thing to look for is options. It is helpful to have different solutions to solve the same problem. One of the most painful parts of implementing a new system is migrating the data from the old to the new system. If you were asked to preform this migration, how would you do it? The last thing I would want to do is manually type in the data. There isn't always a perfect solution. What you choose today might not be feasible tomorrow. A coworker might want to take a completely different approach. A SaaS tool should support several ways to import data. Including:

1. An API suite. If a SaaS tool doesn't have an API to their system then don't buy it. An API will allow a user to important anything and everything. It is the first thing a developer is going to look for data migration. It is the only way to reliably import terabytes of data.
2. A toolbox full of tools. I'm comparing a toolbox to a host of different options for importing data into the system. An array of many one-off file import options. They allow a user to upload data using a comma separated value (CSV), spreadsheets, MS Project, and zip files. Tools are now integrating with email servers. Just email a document to an email address and it will be added to the system. Having a toolbox full of options to import data can turn a expensive data migration problem into a simple copy and paste. It's game over once you can reduce a complex operations into simple steps that you already know how to solve. The next time you are faced with the problem of trying to import data, ask yourself the question: Would you rather be looking for a developer or opening up an excel spreadsheet?
   

When you have a lot of options for importing data it allows you to streamline the process. Importing data will always be tedious and boring. Don't make it harder then it should be. When shopping for an online SaaS tool first look for an API and then look for a toolbox full of different options.

Connecting to a secure LDAP or Active Directory.

There are hundreds of blogs on how to connect to an AD/LDAP directory service, but there aren't a lot on how to connect to a secure one. Java's method for connecting to a secure directory services involves uses a keystore. If your requirements won't allow you to add the certifcate to your keystore then you'll need to dynamically create one using Java Code.

In my application I need authenticate against a user's directory service. It's a shared database setup and we need to be able to connect to hundreds of directory services. Because of this requirement it is impossible to manage the keystore on the app server and constantly add new certificates to them. You can dynamically use keystores; however, must customers don't know how to create them. Below are sudo instructions on how to create a tool that will allow you connect to a secure directory service when you are only given the certificate.

Let's get started...

You will need to implement your own SocketFactory and set it in the environment properties before you call "new InitialLdapContext".

environment.put("java.naming.ldap.factory.socket", "ldap.TestSocketFactory");

To create your "ldap.TestSocketFactory" you'll want to extend SocketFactory and implement the abstract methods. You'll also need to override the public static javax.net.SocketFactory getDefault() and have it return itself.

Example:

Public class TestSocketFactory extends SocketFactory{

private javax.net.SocketFactory _socketFactory;

public static javax.net.SocketFactory getDefault() {

return new TestSocketFactory();

}

public java.net.Socket createSocket() throws java.io.IOException {

return _socketFactory.createSocket();

}

public Socket createSocket(String s, int i) throws IOException, UnknownHostException {

return _socketFactory.createSocket(s, i);

}

public Socket createSocket(String s, int i, InetAddress inetAddress, int i1) throws IOException, UnknownHostException {

return _socketFactory.createSocket(s, i, inetAddress, i1);

}

public Socket createSocket(InetAddress inetAddress, int i) throws IOException {

return _socketFactory.createSocket(inetAddress, i);

}

public Socket createSocket(InetAddress inetAddress, int i, InetAddress inetAddress1, int i1) throws IOException {

return _socketFactory.createSocket(inetAddress, i, inetAddress1, i1);

}

}

In your constructor you will need to do the following:

• Initialize a new KeyStore and CertificateFactory objects

KeyStore store = KeyStore.getInstance("JKS");

store.load(null, "changeit".toCharArray());

CertificateFactory cf = CertificateFactory.getInstance("X.509");

• Load the Certificate

String fileName = You'll need to load this from a property, or theadLocalVariable.

X509Certificate cert = (X509Certificate) cf.generateCertificate(new FileInputStream(new File(fileName)));

• Convert the certificate to a KeyStore

File tempFile = File.createTempFile("certKey", null);

OutputStream out = new FileOutputStream(tempFile);

store.store(out, "changeit".toCharArray());

store.setCertificateEntry("alias", cert);

out.close();

• Load the KeyStore

InputStream in = new FileInputStream(tempFile);

KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType());

keyStore.load(in, "changeit".toCharArray());

in.close();

• Create the SocketFactory

TrustManagerFactory trustMnagerFactory = TrustManagerFactory.getInstance(SECURE_ALGORITHM);

trustMnagerFactory.init(keyStore);

KeyManagerFactory keyManagerFactory = KeyManagerFactory.getInstance(SECURE_ALGORITHM);

keyManagerFactory.init(keyStore, SSODelegate.CERTIFICATE_PASSWORD.toCharArray());

SSLContext sslc = SSLContext.getInstance(SECURE_PROTOCOL);

sslc.init(keyManagerFactory.getKeyManagers(), trustMnagerFactory.getTrustManagers(), new SecureRandom());

_socketFactory = sslc.getSocketFactory();

If you want to ignore validation on the certificate you can do the follow when you are creating the SocketFactory

sslc.init(keyManagerFactory.getKeyManagers(), getOverrideTrustManager(), new SecureRandom());

public static TrustManager[] getOverrideTrustManager() {

TrustManager[] trustAllCerts = new TrustManager[]{

new X509TrustManager() {

public java.security.cert.X509Certificate[] getAcceptedIssuers() {

return null;

}

public void checkClientTrusted(

java.security.cert.X509Certificate[] certs, String authType) {

}

public void checkServerTrusted(

java.security.cert.X509Certificate[] certs, String authType) {

}

}

};

return trustAllCerts;

}