From P. J. Reed
As hinted yesterday, the real project I have to announce is the Bag Database. This is a server that will scan and monitor an arbitrary directory of bag files, index them, and provide a web-based interface that can be used to quickly search through, analyze, and download them. Have you ever wondered, "Do we have any bags that have a TexturedMarker message in them?", or "What did the path this vehicle followed look like on a satellite terrain map?" This will help you answer both of those questions.
It was designed primarily for internal use; our team has a NAS on which we store thousands of bag files, many of which are several GB in size, and searching through them by hand was difficult and time-consuming. With the Bag Database, everybody can still use tools such as Samba or SFTP to put their bags on the NAS, and the Bag DB will automatically analyze them and make them available in its UI. We have about 15 TB of bags, and it takes about half a second to search through all of them for arbitrary message types or topics.
Features:
- Display any of the information about a bag normally obtained through "rosbag info"
- Quickly search for bags based on their filename, location, contained message types, or published topics
- Filter the visible list of bags based on start and end times, latitude/longitude, size, and more
- Store user-entered metadata such as the vehicle name or description
- Display the bag's path of GPS coordinates on a MapQuest or Bing map
- Use Google's reverse-geocoding API to get a string describing a bag's location from its lat/lon coordinates
- Identify duplicate bag files and tell you about them (although this UI could be better...)
The Bag Database is a Java servlet that only needs a PostgreSQL database to be useful, and it's easiest to deploy it as a Docker container. Source code, documentation, and installation instructions are all available on GitHub: https://github.com/swri-robotics/bag-database
I know it still has a few rough spots I can work out, but I thought now was a good time to go ahead and release it and see how much interest there is.
Feel free to submit any issues or feature requests on GitHub, and let me know if you have any other questions about it.
