Looking for help Investigating Tlog file

Joined
Jul 31, 2018
Messages
7
Reaction score
2
Background:

Originally used my Solo for pictures and video using the solo app, and well just fun flying.

In the ast month I have tried using the Tower App to take geo-referenced photos and stitch them together into orthomosaics. I have flown probably a dozen flights like this with my Solo with out incident. Everything has been going pretty smooth, until my last flight which was Aug 5th. I turned on the drone, waited for GPS position, everything appeared good to go in the tower app, 10 sats, battery fulll, telemetry signal all good etc. uploaded my predefined mission flight plan, it says the waypoints were uploaded to Solo. Then I armed the drone, and swiped right again to take off. Solo only got maybe 6" off the ground then rolled over on it's side. All the while propellers are in full take off power mode, one of the propellers breaks in half and then it rolls upside down, at which point solo detected it had crashed and shut down.

I am trying to figure out what if anything I did wrong, so this doesn't happen in the future. Frankly won't be flying the Solo again until I know what happened so I can correct my procedure if there was a flaw, do repairs if something is broken/faulty, etc etc. Other than the broken propeller which was replaced, everything seems to function just fine, I turned on and ran solo, minus propellers to ensure everything worked. I am relatively new to interpreting the tlog files, so thought I would reach out to the community for some help.

If someone would like access to the tlog file I can share, any help would be greatly appreciated.

Investigation.... I downloaded the tlog file and uploaded to mission planner. Tlog file attached here.

First thing I notice is that the mission plan doesn't show up in this tlog like it has in other Tlog files I have looked at.

Just before 69% there is a "bad gyro health" message that pops up in mission planner. There was no indication of this through the Tower app when I was flying.

At ~77% is where I believe I am arming/ taking off - which has me a bit confused becasue mission planner is still saying disarmed at this point. But you can tell it is trying to take off because the horizon rolls over, which is what happened during the flight.

It keeps saying Arm:Leaning through to about 79.5% then it flashes "System shutting down", "Error compass variance" and "inconsistent compass" messages. This is where I picked the drone back up and set it right side up.

Just after 80% the horizontal position of the drone jumps about 100m, which is likely due the drone being upside down and the GPS position would be way out. Then it starts giving a "Bad AHRS" message (pitch roll yaw sensor) It continues to also give an "error compass variance message" through the remainder of the log.



Questions for the community:

Are any of the error messages stated above indications of a faulty or broken part, or is this simply because I did not go through the calibration routines (compass, level) in the solo app immediately prior to this flight? Any time there is significant time lapse between flight I typically run through the calibration, but seeing how I had flown the day before I did not this time. This incident has me thinking it should be calibrated before every flight. If I were to do the calibrations before every flight would that help minimize the risk of a gyro error like this in the future?

Ideas on how to correct this (procedure, hardware etc)?

Should I use another app? Tower seems to be a bit unstable for my liking, but it has all the features I wanted for free, hence it was my first choice. During a few previous flights the tower app crashed but I was able to restart and connect to the Solo while it was still in Auto Mode so I didn't make much of it.

I have read a bit about Solex - would people recommend using Solex rather than Tower for "Survey Missions", is it more of a stable platform to work from? Can it do survey missions?

Thanks you in advance for any assistance.
 

Attachments

  • 2018-08-05 11-00-52.log0wp.zip
    7.5 MB · Views: 6
  • 2018-08-05.zip
    2.5 MB · Views: 7
1) No log attached :\
2) Please attach the dataflash log, as it tells us a bit more than the tlog does: System Logs | 3DR Solo Development Guide
3) Tower is unstable, but its instability would not have/could not have caused the above to happen really
4) Solex is pretty great, but I can't speak to Mission functionality yet. I plan with Mission Planner on PC and monitor with Tower. Trialing a pure QGC workflow, as well as possible pure Solex.
5) Some users have reported stuck IMUs with their Solo, which were fixed by a firm/hard slap on the "belly" of the Solo. YMMV.

If some folks here have their PhDs in reading logs, I'm about Middle School level, so I'll take a crack at it but hopefully this catches the eyes of better folks as well.
 
  • Like
Reactions: uc731
Thank you for the quick reply! I tried uploading the tlog file but it says the uploaded file does not have an allowed extension. Sorry, new to the forum, how do I upload the tlog file?
 
Thank you for the quick reply! I tried uploading the tlog file but it says the uploaded file does not have an allowed extension. Sorry, new to the forum, how do I upload the tlog file?
Easiest way is to zip the log files, which also has the benefit of shrinking the size greatly.

Please upload the dataflash log as well/instead of the tlog.
 
I believe the 11-00-52 file is the one with the incident in question, please let me know if you need anything else
Those files are just the waypoint files. Please read the link I have posted above for instructions on how to get the dataflash logs.
 
Those files are just the waypoint files. Please read the link I have posted above for instructions on how to get the dataflash logs.
Zipped all files together and attached on the original post. As I wasn't 100% sure what you needed I have attached all files from that day. Let me know if you need anything else.
 
  • Like
Reactions: Saijin_Naib
I won't be able to look until I get home. However I can tell you it is incredibly unlikely you did anything wrong. Flipping over in the air and crashing is almost always a mechanical failure of a motor or propeller, which falls under the category of "shit happens." T-Log files can be a bit delayed in what you see on the screen, so don't look too far into the timing of messages displayed.
 
I won't be able to look until I get home. However I can tell you it is incredibly unlikely you did anything wrong. Flipping over in the air and crashing is almost always a mechanical failure of a motor or propeller, which falls under the category of "shit happens." T-Log files can be a bit delayed in what you see on the screen, so don't look too far into the timing of messages displayed.

Thx P2P, that makes me feel a bit better, didn't think I did anything wrong but always nice to hear it from someone else haha

The only thing that really makes sense is maybe one or both of the rotors on the right side didn't power up fast enough causing the left side to over power and hence the roll over to the right, I know the propellers were fine, inspect those religiously before each flight, and i remember seeing the rear right side propeller break when it rolled over and touched the ground. Other than that no visible damage besides a little paint being gone from the metal at the top of the motors.

Is there anyway to test the solo to see if the motors are all functioning properly? I have powered everything up without the rotors on and everything seems to be fine, at the same time I don't want to power it up and have the same thing happen a second time.
 
I won't be able to look until I get home. However I can tell you it is incredibly unlikely you did anything wrong. Flipping over in the air and crashing is almost always a mechanical failure of a motor or propeller, which falls under the category of "shit happens." T-Log files can be a bit delayed in what you see on the screen, so don't look too far into the timing of messages displayed.
I agree. Try flipping Solo over in any mode besides ACRO- it will fly sideways but not onto its back.
 
Didnt even get a chance to really take control and fly at all, hit the take off button and it seemed like 1-2 seconds later it was upside down broken.... crash detected... shutdown.
 
Thx P2P, that makes me feel a bit better, didn't think I did anything wrong but always nice to hear it from someone else haha

The only thing that really makes sense is maybe one or both of the rotors on the right side didn't power up fast enough causing the left side to over power and hence the roll over to the right, I know the propellers were fine, inspect those religiously before each flight, and i remember seeing the rear right side propeller break when it rolled over and touched the ground. Other than that no visible damage besides a little paint being gone from the metal at the top of the motors.

Is there anyway to test the solo to see if the motors are all functioning properly? I have powered everything up without the rotors on and everything seems to be fine, at the same time I don't want to power it up and have the same thing happen a second time.
I wouldnt do anything until P2P can review your logs. The same thing might happen again if it is a mechanical malfunction. If the motors are fine, it might be an issue with the ESC's or a power issue with the main board.

No need to be a "test pilot".
 
  • Like
Reactions: uc731
From what I checked:
Nothing looked amiss with IMU/Compass/GPS/Power.

Motor outputs seemed to be similar to one another, and didn't throw a WARN in the Auto-Analysis.

Anything beyond that is beyond me. Time to watch & learn from others.
 
From what I checked:
Nothing looked amiss with IMU/Compass/GPS/Power.

Motor outputs seemed to be similar to one another, and didn't throw a WARN in the Auto-Analysis.

Anything beyond that is beyond me. Time to watch & learn from others.

Thx, I am starting to think all it was is the calibrations were out... way out! Guess I will just do the calibrations before every flight. Is this a pretty standard procedure for others?

Just to be safe, going to wait for P2Ps evaluation of the logs as well before flying again :)
 
Thx, I am starting to think all it was is the calibrations were out... way out! Guess I will just do the calibrations before every flight. Is this a pretty standard procedure for others?
I make sure I do one really careful calibration (level on a table that is known level, compass when I'm way out in an ag field in the middle of nowhere, no metal/electronics on me), and otherwise fly & forget unless the Solo tells me otherwise. I've had no issues thus far.
 
  • Like
Reactions: uc731
Well this is interesting. Just after taking off, the log shows a massive uncommanded roll to the right. Both right sides motor outputs (1 & 4) went to maximum thrust trying to compensate, but it would seem both of those right side motors were failed.

This first graph shows the pitch, roll, altitude compared to the autopilot's desired pitch, roll, and altitude. You can see it was fun until the roll just dropped to the right uncommanded.
crash1.png


This second graph shows the motor outputs, and I scaled the battery current (amps) to show at the same time. You can the see the motor outputs begin going up to normal takeoff thrust (around 1555) and the current draw ramps up as well. Then at the same time as the uncommanded right roll, you see the right side motor outputs go to maximum thrust trying to stop the roll. But the current draw actually drops down and levels off. The current draw going down at the same time as motors going to maximum thrust shows those right side motors definitely failed and were not producing any thrust. Those motors going to maximum thrust while actually working would have shown a very steep spike in current draw.
crash3.png


So as I suspected, it doesn't appear you did anything wrong. However, it is incredibly unlikely that two motors or ESCs or motors simply died at the exact same time. This could be that ESC defect 3DR refused to fix and patched with firmware slew rate limiting. When it hit the ground, did it appear all 4 motors were running again? If all 4 motors were running, that's probably what it was since there is a "oh shit restart" code as part of the firmware patch. If the motors or ESC keeled over dead, you would have seen them not running and they wouldn't be running today either.
 
This makes total sense with what happened in the field that day, so the only real fix as we talked about is to install the green cube which eliminates the issues with the ESC defect. All motors are functioning fine now by my tests the other day, started it up on the ground no props and everything functioned fine. Will function better with the new mRo GPS chip that just arrived! Going to chalk this one up to what is a very unlikely occurrence. Will post again if I have another issue like this. Thanks again for the info P2P. Cheers!
 
  • Like
Reactions: Pedals2Paddles

Members online

Forum statistics

Threads
13,093
Messages
147,741
Members
16,048
Latest member
ihatethatihavetomakeanacc