Improve message tracing in Exchange online
We have had a lot of issues with spam, whether its cryptovirus emails getting through, or good emails getting improperly blocked. Because of this, we need good message tracing (to find the emails), which we do not feel we have with exchange online. We would like to make the following suggestions:
- Need to be able to trace further than 7 days back without a 4 hour wait per trace. Our previous message tracing system could go back the entire year nearly instantly, but we need at least 30 days without the 4 hour wait per trace. This was pitched as something that could replace our previous tracing system, and this does not do it.
- UTC time is confusing in message tracing. It uses “AM and PM.” AM/PM in UTC time is ambigious since UTC time it used all over the world. So either use UTC, or fix the timezone to use our timezone automatically. It looks like very little effort was put into implementing message tracing if this made it past QA.
- We would like more options in message tracing, especially completely basic fields like searching by subject. I would like to know how this was rolled out as an enterprise product without that a "subject" search field when tracing emails. Part of the reason it is so slow is probably because you have very few search fields to refine.
- Overall speed improvements, even tracing back less than 7 days or less is slow (sometimes "spins" for several minutes", but more than 7 days is 4 hours+). We basically received a poorly formatted spreadsheet after 4 hours.
A premier case was opened for all of this, the features simply do not exist and its not a configuration issue. They did mention compliance center can help search for emails quicker, but does not trace.
Thank you for taking the time to submit this feedback. Since there are multiple pieces and layers of feedback in this single post, it makes it more difficult than many to address. First, let us share a little about what we’ve been doing. Since this post was made, we have prioritized performance and reliability improvements for both Message Trace (inside 7 days) and Historical Search (typically outside of 7 days). We’ve added details to Message Trace that weren’t there before, decreasing the need to run Historical Searches inside of 7 days. For Historical Search, we have improved the results to be more clear for those who are not familiar with the Exchange Message Tracking log format. Additionally, while we get the total value of Message Trace, we’ve also prioritized reducing the constant need to search & destroy. We’ve made tremendous strides in effectiveness, even as the bad guys got more creative and aggressive. We’ve rolled out ZAP (which is the automatic version of ‘search and destroy’), and we’ve improved our compliance search capabilities. Hopefully you’ve noticed these investments.
But, we’re not finished. We’re working right now to address some of the complaints and confusion around the Message Trace admin experience itself. Our strategy is to continue adding more events to the Office 365 Graph API, and provide ways for larger customers to get at larger datasets faster. More than just providing you with raw data, we’re working on Security & Compliance features that reduce the effort involved and focus on specific tasks, like identifying which campaigns we’ve protected you from, and what suspicious activity you might want to beware of.
We also hear the feedback about improving subject based search, improving the retention time of Message Trace beyond 7 days, and continuing to improve performance. While we have nothing additional to communicate specifically in these areas at this time, we continue to welcome feedback and scenarios that helps us prioritize — know that we’re listening and we’ll continue working to improve these scenarios.
I would like the ability to setup an event stream of all message logs to a log analytics workspace. Then I can get stats about NDRs received etc.
Just like the O365 Activity log.
Happy to see there is something like this being done.
Personally I would love to see the subject in message trace asap.
Even if this would mean a higher license plan, it still could be worthwhile for our administration.
(our helpdesk asks for info for their reports)
Wayne Singh commented
Recently (for just over a week) we have been getting legitimate emails being forwarded to our IT@ inbox. Using message trace basically just said spam with no further details. I currently have had Microsoft support looking into this issue for 3 days and they can not work it out either. We need to see what spam rule is triggering this. At the moment it is impossible to work out why a good email is being marked incorrectly.
There is no excuse for the slow search results. I'll grant the need to wait perhaps a minute or so. But several hours? Are you guys searching by hand? Pony up for a couple extra servers already.
You removed a simple view where I could find easily inbound and outbound emails for a user and it's status whether was delivered or not. I feel like this product is going down hill instead of improving.
Now you have to wait long time to see a simple thing.
Kaden Sinclair commented
The slow reports for longer than 7 days is pretty rough. We have a security issue and need it resolved right away. Waiting hours to find information that could protect the company makes hosted Exchange tough as a solution. Being told it could "take up to 48 hours" by support was astonishing. I feel like I should ask for the report by carrier pigeon and on stone tablets. Our on premise server could deliver a report in minutes and allow us to take action. Now, we risk more damage or acting out of hand. Effectively, this causes real damage to a company during a security risk. Our SOC 2 compliance for hosted solutions would never pass with this type of datacenter sluggishness.
In addition to your updates to the time format, i would like to see the time include the microsecond values too. Currently they only show up to the second interval and this means that the rules are often sorted in the wrong order. I.e. they dont show in the order of execution because many events happen in the same second and the list is not sorted by a value of less than 1 second. This unnecessarily makes troubleshooting more difficult.
John-Paul Muldoon commented
#2 point on John's list was in relation to UTC being used for message tracing. From what I can see, this point doesn't seem to have been responded to at all. Feature requests for the ability to use a Tenant's timezone, or some other ability to add a timezone option has been asked for years and years without any significant response or implementation from Microsoft.
When an admin is running a message trace, or many, the last thing he/she is wanting to do is manually convert the timestamps to their respective timezone or explain to management why the times are different.
Our end users cannot wait hours or days for us to trace an email that was sent less than a month ago. This feature needs serious attention as it is currently unusable in a serious situation.
Anon I Mouse commented
When will we see a major overhaul to Message Tracing? As of May 2018 Message Tracing is still an ineffectual component. I don't even use it. To properly trace messages you must be able to trace every word of every header of a message. Not just a select few. And you need to be able to go back MONTHS, not just a few days. Query results, even in the 10's of thousands, should only take a few minutes. And the results should in effect be the message and it's headers, with the option to export the message to it's original format, or EML format. ProofPoint SmartSearch is already years ahead of any message tracing available in Office 365. When this is updated, I sure hope it's not an "E5" only feature....
Message tracking should also include full e-mail headers. Always asking user e-mail heades when they complain about junk mail is a pain. It should be possible to get message headers trought the web tracking interface
There is no improvement what so ever... Historical search (7 Days +) takes at least a day even with couple of mailboxes. Having no search capability with Subject is not acceptable.
Scott Moore commented
Looks like Subject based searches has been added to SCC\Threat Management\Explorer, but this requires E5 licensing. As I noted over a year ago, this should be a fundamental functionality of Message Trace and should not require elevated licensing in order perform this action. Whether it is added to the Message Trace in EAC or the Explorer in SCC is available to all O365 customers regardless of licensing, that doesn't matter, just the basic ability to search by subject is what matters.
Also, I have to agree with other here that >7 day old search performance is in no way improved. This takes FAR too long.
Message tracking on 15 days for 1 user (about 50 email) not yes completed after 23 hours????
The MS support told me that is normal behaviour, it can take 4 days... are you aware that this tool is totally useless and it waist the time of the IT support and also of the user that need support????
Message trace performance on 7+ day old emails is getting worse and takes >11 hours! Totally unacceptable!
+1 CAN'T AGREE ANYMORE! NEED IMPROVEMENT!
Seth Hohensee commented
We get requests for data over a week old very frequently. Getting a spreadsheet several hours later just isn't responsive enough. Also, given the small result and page sizes, subject filtering is a must.
Douglas Plumley commented
Glad to hear you are considering improving subject based searching. The only solution we have today is to dump all our message traces to file/memory and then search by subject over that. From a load perspective that means we might be dumping several hundred thousand messages just to find one message.
It's an unnecessary, hugely inefficient load on your infrastructure and a lot of time wasted for us.
Douglas Plumley commented
Search by subject and direction (inbound/outbound) would be really helpful.
Scott Moore commented
The ability to perform message traces in the EAC GUI by Subject as John noted is a fundamental requirement. Also, the ability to search by sender domain in the GUI should be a fundamental requirement.