Conversion to Movable Type - Part 3 - Assets

This post is the third part of a 3 part series. See also part 1 and part 2.

The proper way, given enough time, would be to reverse engineer (or find documentation for) the MT database format and understand it well enough to construct all of the rows to describe the assets as if I had uploaded them. But I already had a published and tested XMLRPC interface, this is a one off approach, and I was running out of steam working on this, so I used the XMLRPC way to reupload all of the files. But, my custom approach had a few drawbacks:

• I had to reupload all of the files that were already on my server, overwriting them
• Since I was overwriting the files, the modify/create dates on the files were all reset to the time of the import; I couldn't find a reasonable way within my motivation to fix the file timestamps (thus affecting Apache's return) or via MT
  • The table in MT looked like it was easy to fix, but who cares, if the HTTP server still returns the wrong date because the file's date on disk is wrong?
• I had to modify the DeniedAssetFileExtensions in mt-config.cgi to allow the upload of certain files (like exe for my installers) (this was new upon upgrade to 5.11 from 5.04)

For the code, I used the newMediaObject implementation of MoveableTypeImpl from part 2 to actually upload the files. The files, their content and metadata was found simply by walking the filesystem. I wrote some custom code in the SiteScanner object (not shown below) to filter out certain paths that I didn't want to manage as assets in my site. From a high-level view, it's a simple as that, so the rest of the post will be the code. The sitePath is the root of the website, which is also used to determine the relative path needed for the MT interface (i.e. C:\Projects\MyWebsite\tutorials\file.txt to /tutorials/file.txt).

    private static final String FILE_SEP = System.getProperty( "file.separator" );

    private static void importAssets( MovableTypeImpl mt, File sitePath, SiteScanner scanner )
            throws IOException, XmlRpcException {

        long totalLength = 0;
        long totalFiles = 0;
        for ( File file : scanner ) {
            totalLength += file.length();
            ++totalFiles;
        }
        System.out.println( "About to upload " + totalLength + " bytes in " + totalFiles + " files" );

        for ( File file : scanner ) {
            String filePath = convertToSiteRelativePath( sitePath, file );

            //TODO: can I preserve the date?
            System.out.println( "Uploading " + filePath + "( " + file.length() + " bytes)" );
            String dest = mt.newMediaObject( filePath, getFileData( file ) );
            System.out.println( "  Uploaded to: " + dest );
        }
    }

    private static String convertToSiteRelativePath( File sitePath, File file ) {
        String filePath = file.getAbsolutePath();
        if ( file.getAbsolutePath().startsWith( sitePath.getAbsolutePath() ) ) {
            filePath = filePath.substring( sitePath.getAbsolutePath().length(), filePath.length() );
            if ( filePath.startsWith( FILE_SEP ) )
                filePath = filePath.substring( FILE_SEP.length(), filePath.length() );
            filePath = filePath.replace( FILE_SEP, "/" );
        }
        return filePath;
    }

    private static byte[] getFileData( File file ) throws IOException {
        int total = 0;
        int length = (int) file.length();
        byte[] ret = new byte[length];
        FileInputStream reader = new FileInputStream( file );
        while ( total < length ) {
            int read = reader.read( ret );
            if ( read < 0 ) throw new IOException( "Cannot read all data from " + file );
            total += read;
        }

        return ret;
    }