Does the test need an update?

Forums Benchmark Discussion Does the test need an update?

This topic contains 10 replies, has 3 voices, and was last updated by Admin 1 month, 1 week ago.

Viewing 11 posts - 1 through 11 (of 11 total)
  • Author
    Posts
  • #1191

    NL-og
    Participant

    What do you guys think does the test need to be improved or is it still up to date?

    #1193

    Admin
    Keymaster

    What would you change?

    #1270

    thermal
    Participant

    I wanted to try this, but Logic could not find the Boom Room.SDIR impulse response for Space Designer nor the audio file 3hook96.wav (part of the EXS instrument FM Short SQ.exs). Would probably be a good idea to make a self-contained project file.

    #1271

    Admin
    Keymaster

    I remember I had that too, but somehow I fixed it. I don’t know if these settings were removed from the plugins or if their files were just located somewhere else because of a change in an update.

    What do you guys recommend changing it to? It has got to be something similar to not make all the old test results obsolete. Will any setting from a same plugin use the same CPU power?

    • This reply was modified 6 months ago by Admin.
    #1321

    thermal
    Participant

    Maybe you should join forces with this guy, the test worked right out of the box and was very easy to use πŸ™‚

    https://www.logicprohelp.com/forum/viewtopic.php?t=138612

    #1322

    Admin
    Keymaster

    That looks good! I’ll get in touch with him and see if we can use this. πŸ™‚ Thank you.

    #1436

    Admin
    Keymaster

    I haven’t been able to reach the guy from Logic Pro Help yet, so I decided I would try to fix the test myself. Basically, the problem with the IRs that cannot be found should be solved now. I selected an option that includes them inside the project file, no matter if you have downloaded them before or not, so everyone should have them now.

    Can you guys check this out for me? You can download the test here. I’ll be happy to hear if the problems still persist or not. πŸ™‚ Thank you.

    #2366

    thermal
    Participant

    Hey, so the test seems to be working well, but it does not cater for a realistic working scenario IMHO. In testing my newly acquired 2018 Mac Mini I attained the best score using the lowest buffer available (32).

    Working on a real project, 32 is not workable at all except in the initial writing phase. I do quite a lot of buss processing and use lots of hungry 3rd party plugins (such as Acustica Audio ones) and inevitably end up with at least 512 (and usually 1024) buffer towards the end of the mixing stage.

    So my proposition is to create an additional benchmark test which combines software and audio tracks using latency-inducing plugins on individual tracks, auxes/busses and the master buss. Maybe even throw in lots of automation on the heavy plugins for good measure. That way we would get a lot closer to a kind of “minimum performance” (“this mac can at a minimum handle this much”) benchmark number. I’d be happy to assist in setting this up if desired.

    edit: Just to be clear, I am not thinking of using any 3rd party plugins in the benchmark template.

    #2371

    Admin
    Keymaster

    That sounds like a good idea. I had the same “problem”, having my highest scores with Buffer at 32. The only issue I see with this plan is, what to do with all the existing benchmark scores? They’ve been added over the past years and have consistency because everyone ran the same test. The original test (Evan’s Benchmark Test) goes back to ten years ago and was shared on many sites, meaning that a lot of people had run exactly the same thing.

    Another question is also: Will the gaps/margins between the various Macs differ in size or will these spaces be the same while all scores shift up or down in overall? Do you know what I mean? In the end, this is probably the most important thing, because that’s what makes the models comparable to each one in the first place.

    #2388

    thermal
    Participant

    The best thing would probably be to use a fresh table, kind of like Geekbench 4 versus 3. So the results could be presented like:

    Mac model – Simple benchmark score (lower buffer = more tracks) – Complex benchmark score (higher buffer = more tracks)

    Mac mini i5 3.0 GHz (2018) – 135 – 50

    or something along those lines. The overview should have an introduction clearly stating the differences between the simple and complex benchmarking.

    Just brainstorming πŸ™‚

    #2389

    Admin
    Keymaster

    Yes, that would be a possibility. I’ll put this on the list of possible things to add! For the moment I’ll give priority to things like changing the arithmetic mean to the median however because, in the end, it’s sadly also a financial question. πŸ™ If the site grows then everything is possible, haha.

Viewing 11 posts - 1 through 11 (of 11 total)

You must be logged in to reply to this topic.