Automated log download script for Solo

Pedals2Paddles · Jul 13, 2017

Want a much easier way to download log files from your Solo and controller? I wrote this little batch file that utilizes WinSCP's console commands to automate it.

The file is called Solo_Log_Sync.bat. I put it in a zip file on the github site. (SoloScripts/Solo_Log_Sync.zip at master · ArduPilot/SoloScripts · GitHub).

Simply run this bat file and it will do all the work for you. First it will grab the tlogs off the controller. Then it will grab the useful logs off the solo. This includes the dataflash logs, rc logs, shot logs, and python log.

It uses a sync command.
- New files are downloaded
- Changed files are overwritten
- Unchanged files are ignored
- Files with no match are ignored
- Nothing is deleted.

You no longer need to have WinSCP installed for this to work. The necessary files are included in the zip and it will just work.

CanadianSolo · Jul 14, 2017

cool i will give it a shot. Cheers bud

Pedals2Paddles · Jul 14, 2017

It occurred to me today that I can just put the WinSCP executable file in the zip and it will just use that. No need to actually download and install WinSCP or change directories to look for it. I've updated the zip to include this.

XevetS · Jul 16, 2017

As usual nice work Matt, works perfect thanks again for your hard work!

CanadianSolo · Jul 16, 2017

Pedals2Paddles said:
It occurred to me today that I can just put the WinSCP executable file in the zip and it will just use that. No need to actually download and install WinSCP or change directories to look for it. I've updated the zip to include this.

haha ya thats what i did. You can download the portable version

REDZMAN · Dec 12, 2017

Pedals2Paddles said:
It occurred to me today that I can just put the WinSCP executable file in the zip and it will just use that. No need to actually download and install WinSCP or change directories to look for it. I've updated the zip to include this.

Using this for some workflows we have built and am wondering what we'd have to change/add to the script to get it to delete all the logs stored on the Solo's side after it downloaded them..

Pedals2Paddles · Dec 12, 2017

REDZMAN said:
Using this for some workflows we have built and am wondering what we'd have to change/add to the script to get it to delete all the logs stored on the Solo's side after it downloaded them..

Presuming you only want to delete the dataflash logs (*.bin) files? Pretty straight forward. It's currently using the WinSCP synchronize command. What you'd want is using the MOVE command instead of synchronize on the dataflash directory.

Currently, the code in the batch file looks like this:

Code:

winscp.com /command ^
    "open sftp://root:[email protected]" ^
    "synchronize -filemask="3dr-rc*;3dr-solo*;python_stderr.log;shotlog.log*;/log/dataflash/*.bin" local "%local_dir%" "/log/"" ^
    "close" ^
    "exit"

You'd replace it with something like this. I'm at work and not able to test or tweak this. But this should be close. Specifically the mv line syntax is what probably needs to be tested and tweaked.

Code:

winscp.com /command ^
    "open sftp://root:[email protected]" ^
    "synchronize -filemask="3dr-rc*;3dr-solo*;python_stderr.log;shotlog.log*" local "%local_dir%" "/log/"" ^
    "mv /log/dataflash/*.bin" "%local_dir%" "/log/dataflash/"" ^
    "close" ^
    "exit"

REDZMAN · Dec 12, 2017

Pedals2Paddles said:
Presuming you only want to delete the dataflash logs (*.bin) files? Pretty straight forward. It's currently using the WinSCP synchronize command. What you'd want is using the MOVE command instead of synchronize on the dataflash directory.

Currently, the code in the batch file looks like this:

Code:

winscp.com /command ^ "open sftp://root:[email protected]" ^ "synchronize -filemask="3dr-rc*;3dr-solo*;python_stderr.log;shotlog.log*;/log/dataflash/*.bin" local "%local_dir%" "/log/"" ^ "close" ^ "exit"

You'd replace it with something like this. I'm at work and not able to test or tweak this. But this should be close. Specifically the mv line syntax is what probably needs to be tested and tweaked.

Code:

winscp.com /command ^ "open sftp://root:[email protected]" ^ "synchronize -filemask="3dr-rc*;3dr-solo*;python_stderr.log;shotlog.log*" local "%local_dir%" "/log/"" ^ "mv /log/dataflash/*.bin" "%local_dir%" "/log/dataflash/"" ^ "close" ^ "exit"

We are downloading the Tlogs and Bin logs for our missions, so I'd think it is both. the Tlogs get submitted if there was a problem during the flight, the Bin logs we are using camera trigger messages to geotag our photos.

Thanks for getting back to me so quickly. I'll show this to a few other folks that were looking into it. If you do get a chance to test it let me know if you would. The other guy that is really interested in this modification is on leave for several weeks and I won't be able to test it for a bit either.

pulsar110 · Dec 12, 2017

I’m watching too as I tried to figure it out but I’m not that smart in this little world. Appreciate your time on this P2P.

pulsar110 · Dec 14, 2017

Hey Matt I gave that a run and this is what it said....
I have worked on it a bit the last few months but havent been able to make it delete the BIN files after downloading. Honestly I know just enough to be dangerous on this one.

I also tried using the full word "move" and it didnt go.

just_bruce · Dec 14, 2017

I think the last "/log/dataflash/" on the mv line is what is causing the error. Try removing it.

pulsar110 · Dec 14, 2017

hey Matt I think I figured it out. I used your line and changed mv to rm. It seems to work.

"rm /log/dataflash/*.bin" "%local_dir%" "/log/dataflash/"" ^

Pedals2Paddles · Dec 14, 2017

rm is remove. If you just do that, it won't actually copy the files from the solo first. If you leave the dataflash as part of the synchronize line, then that would be fine actually.

pulsar110 · Dec 14, 2017

I just changed the line that you put in. It downloads the files and then removes them from the autopilot. I ran it, then used WinSCP to copy them back onto the AP and ran it again. It seemed to work for me. Gonna have a couple others give it a run and make sure its working the way I think it is. It looks like this now:

winscp.com /command ^
"open sftp://root:[email protected]" ^
"synchronize -filemask="/log/fmv/*.txt;/log/dataflash/*.bin" local "%local_dir%" "/log/"" ^
"rm /log/dataflash/*.bin" "%local_dir%" "/log/dataflash/"" ^
"close" ^
"exit"

Pedals2Paddles · Dec 14, 2017

In the code I posted, look at the sync line. I removed dataflash from the sync line, and used the move command in the new line below it instead. In your's you left the dataflash in the sync line, and then remove the files form the solo. Accomplishes the same thing, without my broken syntax