Whilst the flashy front end features of the Qlik Sense February 2021 release have been covered by my colleague, Chris, I wanted to go through some of the changes in one of the increasingly-important back-end tools that Qlik are now developing apace – Qlik DataTransfer (QDT). In my first post on QDT back in September 2020, I went through the basic features so in this one I’ll focus just on the changes in this latest release.
QDT is becoming increasingly important as customers start working with Qlik’s SaaS offerings – either as a new customer or part of a migration. It’s the “missing link” to get on-prem data into the cloud with minimal effort (and zero licence cost – it’s free!). There are much more full-featured tools that do this at full enterprise scale – such as Qlik Replicate – but QDT is a purpose built for this and works really well.
Qlik DataTransfer February 2021: The Headlines
The two big features in the February 2021 release of QDT are:
Bigger files
Described in the release notes as “resumable file upload”, this is a big one for customers with a lot of data. Previously there was a limitation on file uploads of 500MB meaning you’d have to split out any QVDs that were bigger than that into chunks – usually this would mean saving separate QVDs for years, months and/or days. Now it’s possible to upload files up to 6GB including support for resuming those uploads if they fail part-way.
I’d still recommend breaking larger files down into chunks as the upload will be quicker, meaning any updated data will be ready sooner in SaaS but at least this means if you have some QVDs hovering around the old 500MB limit you don’t need to be concerned that your processes are suddenly going to start to fail.
Alternate delivery folder
You can now select to have QVDs created by Data connection queries and Qlik apps stored to a custom local folder. This used to happen automatically to a default folder but now this can be set up as a reload on its own with a custom path:
The use case here is if you want to back these up or upload them to a cloud file service (think OneDrive / Dropbox etc) for access by Qlik Sense SaaS (or potentially another system).
The Details
Whilst the above two new features are nice – especially the bigger files one – I think the angel is in the detail here, to coin a new phrase. There are a host of small changes that are going to make the tool easier to use for Qlik’s customers, here I pick out some of my favourites
- Additional filename character support. This used to be annoying if you had a file called something like “Targets & Budgets.qvd” the upload would fail. You’d have to change your process of creating & loading the QVD to make sure it didn’t use & (or + or # at well at least).
- Clearer cloud app names. In Qlik SaaS, often you have apps with the same name in a Shared/Personal Space as well as in a Managed Space – this meant choosing the right app in QDT by name only could be tricky! Well, now there’s much more clarity with things grouped by space:
- Custom prefixes. In previous versions, any uploaded QVDs would be prefixed by the name of the Data Connection, App or Data folder from QDT. From this release you can select a custom prefix for each configuration item:
- Latest Load Status indicator & run in background. You used to be able to see when a load last ran but not whether it succeeded or failed. Also, if you ran a task manually in QDT you’d have to wait for it to finish before being able to do things in the application. Now these processes run in the background and you can see the live status:
- Showing Last & Next executions for Scheduled uploads. This used to show some confusing configuration string rather than something human-readable. Now we can easily see when things are next scheduled to run and when they last did:
Then there are a swathe of bug fixes and security enhancements as you’d expect.
My wishlist
Whilst these new features are great, there are still a few key gaps I see. I’ve logged the following on the Qlik Ideas board so feel free to go upvote them if you want to see them delivered!
- Why not allow editing of Qlik apps within the interface? QDT already has the whole Qlik Engine installed with it – how hard would it be to bundle a client in there like Qlik Sense Desktop? At the moment the way of creating and managing apps is clunky as you have to have Qlik Sense Desktop installed (which can’t be done on a server OS) to work with them since only QS Desktop supports embedded Data Connections which are required for QDT to work. This seems like unnecessary complication and redundancy.
- Task chaining. Whilst you can select multiple apps to reload within a scheduled reload, you can’t control the precise sequence of them, nor can you set a task to trigger when another finishes. Some form of this would be good for when you have apps that are dependent on each other. For now we manage this by manually scheduling them with a time gap, but this isn’t very efficient.
- Date formats. I know it’s a little thing, but please use the system date format or allow users to customise it or at least use the ISO 8601 standard (YYYY-MM-DD) – I find the American system very hard to interpret!
Qlik DataTransfer February 2021: Conclusion
All-in-all a very solid release in terms of features and fixes, demonstrating Qlik’s commitment to this product as part of their movement towards SaaS as their primary offering. Hopefully they carry on in this vein with more releases over the coming year and tick off some of my wishlist items!
By Alex Walker
Update: Potentially Breaking Behaviour after Upgrade
Just to note that as our customers have started to upgrade to this version of Qlik DataTransfer, we have noticed an issue where the files uploaded by the Data Folders functionality are now prefixed by the folder name on disk not by the Data Folder name in Qlik DataTransfer as it was previously. The recommendation is that you use the new Prefix functionality to set each one to the same as the Data Folder name to avoid any unexpected changes in behaviour or duplication of files. You can read more about this in the Qlik Knowledge Base article on the matter.
Want to find out more about Qlik DataTransfer? Join us for a demo and Q&A session on 22 April – you can sign up for free here.
Comments