You are correct that the MIDI File Specification says Tempo messages must be in the first track of a Format 1 MIDI file.

However, as you have discovered, some MIDI file software gets this wrong. Some possible options you could consider are:

• If you refuse to open the incorrect file, or if you open the file but discard the incorrect Tempo messages, you will probably just annoy people that want to use your program. Users will think your program isn't working right because the file will open and the tempo will work as they expect in other programs.
• If you silently accept or fix the incorrect Tempo messages, users will be happy, but no one will know there was a problem.
• If you show an notification that there were incorrectly placed Tempo messages, but then move them to the first track as you open the file, users will probably be a little annoyed or confused by the notification message, but notice that the MIDI file seems to work. Technical users may investigate and understand what the problem is and realize the other software that made the MIDI file used non-standard placement for the Tempo messages.

...

For some comparison about what other MIDI software does, I have some notes from 2012 that I tested several MIDI apps on my computer and found most of them will use Tempo messages that are not in the first track:

Uses tempo events in a track besides the first track?

Uses
  Sekaiju (gives option to fix, or else uses but prevents changes*)
  Sythesia
  Winamp
  Notation Player
  vanBasco's Karaoke Player
  MidiJam
  Windows Media Player (wmplayer.exe)
  Music Studio Producer 1.23 and 1.25

Doesn't use
  Media Player (mplay32.exe)
  MidiYodi

* I tested Sekaiju 8.2 again now to describe the following details: If you open a Format 1 MIDI file that has Tempo messages that are not on the first track, Sekaiju will ask you if you want it to fix the incorrectly placed messages. If you click Yes, it will move the incorrectly placed Tempo messages to the first track. If you click No, it will keep the messages in their original places, but not allow editing or saving the file, the incorrectly placed tempo messages will be used (the tempo values will change the speed of the playback and the metronome clicks), but the tempo display at the top will remain at the default value of 120.

Do you think some midi will be wrong if I use all tempo tracks. I mean maybe there are some midi that only sound good they are ignored..

I don't have any additional info about playback, but I have found  that when using Anvil Studio's "repair" option to try to fix a MIDI file, it will sometimes incorrectly move tempo and time signature events out of the first track and in to the second.  Super annoying when it does this, and Sekaiju was how I accidentally found out it was happening.

I think a notification would be a good idea if the issue is detected.  And a moving option also good to have.  Personally, I'd rather have a broken file fixed (after asking me) than just have it not work.