Gillius's Programming

Conversion to Movable Type - Part 3 - Assets

The last challenge in the Gillius.org conversion was assets. At first, I tried to find an external solution like the "Asset Handler" plugin, but it only works for MT4. In the end I wrote my own Java code to scan my assets and upload each of them via the XML-RPC interface.

This post is the third part of a 3 part series. See also part 1 and part 2.

The proper way, given enough time, would be to reverse engineer (or find documentation for) the MT database format and understand it well enough to construct all of the rows to describe the assets as if I had uploaded them. But I already had a published and tested XMLRPC interface, this is a one off approach, and I was running out of steam working on this, so I used the XMLRPC way to reupload all of the files. But, my custom approach had a few drawbacks:

For the code, I used the newMediaObject implementation of MoveableTypeImpl from part 2 to actually upload the files. The files, their content and metadata was found simply by walking the filesystem. I wrote some custom code in the SiteScanner object (not shown below) to filter out certain paths that I didn't want to manage as assets in my site. From a high-level view, it's a simple as that, so the rest of the post will be the code. The sitePath is the root of the website, which is also used to determine the relative path needed for the MT interface (i.e. C:\Projects\MyWebsite\tutorials\file.txt to /tutorials/file.txt).

    private static final String FILE_SEP = System.getProperty( "file.separator" );

    private static void importAssets( MovableTypeImpl mt, File sitePath, SiteScanner scanner )
            throws IOException, XmlRpcException {

        long totalLength = 0;
        long totalFiles = 0;
        for ( File file : scanner ) {
            totalLength += file.length();
            ++totalFiles;
        }
        System.out.println( "About to upload " + totalLength + " bytes in " + totalFiles + " files" );

        for ( File file : scanner ) {
            String filePath = convertToSiteRelativePath( sitePath, file );

            //TODO: can I preserve the date?
            System.out.println( "Uploading " + filePath + "( " + file.length() + " bytes)" );
            String dest = mt.newMediaObject( filePath, getFileData( file ) );
            System.out.println( "  Uploaded to: " + dest );
        }
    }

    private static String convertToSiteRelativePath( File sitePath, File file ) {
        String filePath = file.getAbsolutePath();
        if ( file.getAbsolutePath().startsWith( sitePath.getAbsolutePath() ) ) {
            filePath = filePath.substring( sitePath.getAbsolutePath().length(), filePath.length() );
            if ( filePath.startsWith( FILE_SEP ) )
                filePath = filePath.substring( FILE_SEP.length(), filePath.length() );
            filePath = filePath.replace( FILE_SEP, "/" );
        }
        return filePath;
    }

    private static byte[] getFileData( File file ) throws IOException {
        int total = 0;
        int length = (int) file.length();
        byte[] ret = new byte[length];
        FileInputStream reader = new FileInputStream( file );
        while ( total < length ) {
            int read = reader.read( ret );
            if ( read < 0 ) throw new IOException( "Cannot read all data from " + file );
            total += read;
        }

        return ret;
    }