Agreed, I wish that the tests would have been referenced against one or more of the ICP tests
@a3000gt
5 жыл бұрын
you should use standard deviation instead of variance for consistency. it gives a better representation of how consistent the readings are compared to just the range of what was observed.
@adrianr514
5 жыл бұрын
a3000gt but then someone would have to take stats101. 😂
@Day_Paints
5 жыл бұрын
That actually kinda triggered me.
@chrisb.5570
4 жыл бұрын
I know you guys don't sell them and they are often maligned in the hobby, but I would like to see API included in the test kit videos. It would provide valuable information as the bottom of the line in cost/accuracy and what many start with in the hobby.
@johnson4523
5 жыл бұрын
Using an icp or other comparable lab based test as your control would’ve be more appropriate that an average of all of them. Further standard deviation would’ve been a better way to analyze your data than simply looking at the range. As an engineer, how you analyze your data is just as important as how you collect it. Nonetheless good work!
@johnson4523
5 жыл бұрын
Also doing a t test on the data would show if there is any real significance between data sets, although that may be out of scope:)
@skzion2
5 жыл бұрын
Anthony Posyton Yeah, then they'd need to understand the concept of a standard error. 😉
@jacoblytle333
3 жыл бұрын
Yes! Students t-test, but they didnt even make to the point that the could. Heck if they had a good methodology, I would have run the t-test for them.
@gbj6581
5 жыл бұрын
What would have been great would be to have sent those same water samples for ICP testing to a couple of different icp companies. Thus helping on the accuracy portion since the baseline could be way off as its just an average and nota definitive fact that its correct values.
@ReeferGil
5 жыл бұрын
Love Red Sea! Thanks for the experiment.
@12GallonNano
5 жыл бұрын
Averaging results to come up with an "accurate" number is such bad science. They could all be over or undershooting the true value. Calibration / Reference solution with known values that all the tests use as a sample would show how close each can get to that known value. Consistency being based of the average of 5 different people doesn't seem like the right word either. Randy using the same Aquaforest, Nyos, Hanna test 5 times and getting exactly the same results means it gives consistent results. Another reefer may be 50-100ppm higher but hits the same number all 5 times as well. In this scenario it just shows the test is maybe a little hard to follow and has easy user error mistakes. The color shift is being interpreted consistently but the exact correct hue is up for debate. I'd hope BRS can redo this Investigates episode sometime in the future.
@CalebChapman12
5 жыл бұрын
Great testing and experiment! Try it again with a bunch of trident calibration bottles.
@adrianr514
5 жыл бұрын
#1 problem is that most people do not account for the meniscus when measuring the sample water which will skew the results.
@truereefkeeping6022
2 жыл бұрын
This right here
@solarchuva7042
5 жыл бұрын
There are some variables to take into account. 1st the accuracy of each test. Hanna clains +-0.3dkh, redsea +- 0.14 dkh, Salifert +-0.1 dkh. Second you should use a reference certificated solution, consider the kit expiration date, price and of course discuss the importance of the "true value" and what is a tolerance that a hobbist can deal with. To me an alk of 7.0 or 7.3 dkh is the same and is 7is. I would love BRS sell the powders for we make the reference solutions like alk, mg, ca at home that is very simple and only requires an analitical weighter and a graduated flask
@nsillej
5 жыл бұрын
This interesting information but I really wish that this was performed with a Known Standard rather than "random" tank water.
@TattooedDancer91
5 жыл бұрын
@@BRStv Tropic marin sells a Multi Standard solution you can use as reference.
@TattooedDancer91
5 жыл бұрын
@@BRStv then mix it up yourself, use math to make a solution with nice even numbers.
@kylepagel9360
4 жыл бұрын
This
@Mike__B
5 жыл бұрын
A couple comments on the video, first referring to the "accuracy" of a test kit by comparing it to the average is really wrong, like supremely wrong, just look at the Lamotte results and how absurdly high they were which ends up bringing the entire average up quite drastically. A "better" way to do it is to use one of said "costs more than your whole reef" testers which you seem to have (shown in video?), and while that doesn't mean it's more accurate, it does reduce the level of uncertainty related to the test kit itself when comparing lab grade equipment to hobby grade. Second, just as a point of visual emphasis maybe color code the individual people tests, like I know who Randy and Ryan are and they represent the "middle of the road" aquarists but I already forgot the names of the "pro" testers, so maybe as an individual I can maybe give more weight to the numbers that the "pros" got compared to the average person.
@jwhall3
5 жыл бұрын
I agree. I would also add that measuring consistency among all different users is not the same as measuring consistency among one user performing the tests 10 times. The later being a much more useful and important metric. I love what you guys do, but this might be the first video I actually dislike.
@tukanxx
3 жыл бұрын
Please make a new comparison including ICP test and Mastertronic and Reefbot devices.
@jaredstroberg286
5 жыл бұрын
The only way this works is if you know the exact levels of what you are testing. Who knows technically the Lamontte could be the most accurate
@alexvids9232
2 жыл бұрын
yup
@mcmanustom
5 жыл бұрын
It would have been nice to run the same water sample against ICP-OES to see if your base line aligned with ICP testing. How close does mean line match machine analysis?
@joe0813
5 жыл бұрын
should have definitely compared these to a ICP test.
@davidmiller5832
5 жыл бұрын
A video on lab grade units would be helpful too. I realize the cost for such units are much higher than hobby kits, given the overall cost of most enthusiasts reef systems, I believe there is an audience for the information/comparison of whats available to them. The cost of a decent tester is about the same as high end lighting fixtures. So the market is there. I believe the general reef enthusiast doesn't consider $1k to $2k on a lab grade tester not just because of the cost, but more likely because it just isn't a topic of discussion. Personally, I chose to invest in a used Hanna lab multi tester before I invested in an apex controller. I felt the need to have a clear insight into the water quality supersedes the convince of automation. Of course having both is an overall goal.
@jacoblytle333
3 жыл бұрын
The Hanna HI931 is a fantastic automatic titrator. I use one in our lab at work and can say that it is the real deal.
@duckslinger999
5 жыл бұрын
Yo Ryan, what the heck are you doing when testing??. Thanks for the video BRS.
@anannyosamayel1350
5 жыл бұрын
It says Ryan on the screen when u said u were randy
@Flownon
5 жыл бұрын
Keep the awesome thumbnails coming. 🤷♂️
@freemansaquatics5326
5 жыл бұрын
Honestly bi weekly water changes is doing it for me....But I'm only messing with softies ;)
@jameswithers3066
2 жыл бұрын
Can I use the Hanna alkanity tester for both my freshwater and saltwater fish tanks? Do you really need to buy 2 different ones? Thanks in advance
@BRStv
2 жыл бұрын
For alkalinity, Hanna makes a specific checker for fresh and saltwater, so you'll need to use the one that corresponds to the type of water you're testing.
@Percula37
3 жыл бұрын
Be nice to see 5x Trident tests along with these...
@jacoblytle333
3 жыл бұрын
So not only did BRS use tank water and not a standard solution for each test, they didnt even have a lab grade, calibrated piece of equipment to find the true value and compare all the tests to it. This test was ok.... and probably the best I would expect from someone without a heavy chemistry background. But what they should have done was used a standard solution to test the accuracy of the test kits. A standard solution is prepared following a rigourous analytical procedure to make sure that the concentration of the analyte is known. Then this solution is used to determine the accuracy of the test kit. They used tank water and averaged the results of each. The accuracy of all the tests could have been shifted by 100 ppm (not likely, I know) and we would have never known. If they wanted to use tank water instead of a standard solution to give more "real world results," they should have used some form of a control, ie. a piece of testing equipment with known variance and error that has been calibrated using a standard solution as a reference to compare the tests to. The only thing they really tested was the kit's precision. But they really didnt even do that correctly either, because each person only performed the test once. So we dont know how much variance there is within each person performing the test, and thus it messes up the calculation of the overall variance of each test kit. I'm a little disappointed in the video.
@bigadventure3797
3 жыл бұрын
Where can I get that Stylo at 2:52? Is that a Rainbow Stylo? Wow that;'s nice
@BRStv
3 жыл бұрын
I can't remember exactly, but the corals in the 160 were originally stocked by WWC, Austin Aqua Farms, and BattleCorals, so it's likely from one of those three vendors 🙂
@DC-xg6gt
5 жыл бұрын
any chance you guys can add some of the other common testers out there like Alkatronics, KH director and KH guardian?
@BRStv
5 жыл бұрын
The KH director is the only one of those three that we carry, so while nothing is planned right now, I certainly wouldn't rule it out 🙂
@rosedonnapur
4 жыл бұрын
I’m dyslexic I’m just wondering if there’s a merchene I can just leave in the corner of my set up that records pH serlinaty and outher stuff.
@BRStv
4 жыл бұрын
The Apex is a great tool for reading and recording data like pH and salinity if it's within your budget.
@hotdroper
5 жыл бұрын
Good video but the results are misleading. As others have pointed out average doesn’t tell you anything. You need a reference value, either with a very high end kit or by mixing your lab grade salt and using that as a standard or using an off the shelf calibrated standard solution. Standard deviation is really the important thing here because some of these kits provide results that span all over the range and should never be used. That can be useful without knowing the true value of alk ca and mg. Really what we’re seeing is that all these kits suck, some just suck more than others. Also can you use different kits off different batches?
@Super21123
5 жыл бұрын
Is there a video on testing to get best results?
@donlocke64
4 жыл бұрын
so are you saying we need to get different brand test kits for different chemical tests
@BRStv
4 жыл бұрын
The moral of this story is that you should pick a test kit that is easy for you to perform and one that you can get consistent results with
@bradmckay8916
4 жыл бұрын
Should I get aqua forest kit?
@lovermansmith9082
5 жыл бұрын
Good job
@alexvids9232
2 жыл бұрын
fail, what if lamotte was right? you cant know by guessing.
@TheFugsTV
5 жыл бұрын
If it's accurate, would that not make it consistent?
@nc474
5 жыл бұрын
So which brand should i get as a new saltwater reefer
@BRStv
5 жыл бұрын
We use a lot of Red Sea brand kits around here. That said, we do typically use the Hanna checker for alkalinity as well as phosphates.
@matyas4878
5 жыл бұрын
*Lamotte has left the chat*
@Kevkevkev763
5 жыл бұрын
Matyas Castro lol
@skzion2
5 жыл бұрын
Matyas Castro I'm wondering how Lamott could have fared so poorly. I'm inclined to take its methods very seriously. It's interesting, though, that its approach may require a more skillful tester.
@PokePackFire
5 жыл бұрын
this feels like a sports race.
@NinjaSushi2
2 жыл бұрын
How do you test if you're colorblind?
@BRStv
2 жыл бұрын
I believe there are some online tests that you can take. If you have further questions, I would suggest reaching out to an expert on eyesight like an ophthalmologist
@jamieliniger161
4 жыл бұрын
Do you use to test other parameters as KH,Ca and Mg?
@BRStv
4 жыл бұрын
Alkalinity, calcium, and magnesium are most commonly tested in a reef tank, but it's not a bad idea to test other parameters too. In particular, nitrates and phosphates.
@jamieliniger161
4 жыл бұрын
@@BRStv ok Thank you
@H_A_F_GAMING
4 жыл бұрын
Wheres my dam brand API
@BassBulletinBoard
3 жыл бұрын
Why would they make test kits that dont work? Sounds like a lawsuit to me
@BRStv
3 жыл бұрын
It's less about the test kits not working and more about which test kits are more accurate or easier to use than others. The mantra "right tool for the right job" definitely applies here. If you need an ultra accurate reading of a certain parameter, you may choose one test kit vs another. If you need just a general ballpark, you may choose a different kit.
@truereefkeeping6022
2 жыл бұрын
I actually got rid of the trident because of how inaccurate it was it never matched any test kit and the calibration fluid that comes with the reagents is terrible and not accurate at all which makes the trident inaccurate even if you use your tank water to calibrate the trident based off of your test kits and your tank water it still doesn’t match up like it supposed to and it changes overtime the worst $600 purchase I ever made
@rossauce12
5 жыл бұрын
why not do api, while not known for accuracy.. probably the most used in the hobby
@BRStv
5 жыл бұрын
We've only ever done a video on a product that we don't carry once and won't do it again. We've looked at the API test kits in the past and our product team decided that they don't fit our good, better, best lineup.
@octapusdude629
5 жыл бұрын
What about Strontium and potassium?
@BRStv
5 жыл бұрын
We haven't tested any strontium or potassium kits just yet, but that would be interesting for sure!
@StNovaSt
5 жыл бұрын
Where was API? Most if not everyone started out with it and many still use it. C'mon BRSTV, think about the beginners too.
@BRStv
5 жыл бұрын
While it's fair to say that a large number of reefers do start out with lower cost kits like the API version, we have only tested a product that we don't carry once and won't do it again. We'd be jerks to publicly review items that we've looked at and decided not to bring in for one reason or another.
@StNovaSt
5 жыл бұрын
@@BRStv You wouldn't be jerks. You'd be respected. What I just learned is that this was just marketing. I love BRS but this didn't feel like it was about us reefers. Felt like it was mainly about your suppliers. If it's about reefing, it has to be more open. Should be more about what we use. Not what you sell. If you sell it and present good reasons you sell an item through comparison, we will throw our money at you. Just sayin.
@PokePackFire
5 жыл бұрын
dear brs, i see a lot of lab grade products, but where are your lab coats?
@BRStv
5 жыл бұрын
Actually, we'd consider all of the test kits in this video to be hobby grade kits 🙂
@PokePackFire
5 жыл бұрын
@@BRStv i mean can you sell us some medical jackets with your logo?
@MaxHedrm
5 жыл бұрын
That's not Accuracy, that's merely the closest to mean. Assuming the average of a variety of test kits is the "correct" answer is dangerous given the ranges of these test results.
@tsurro8086
5 жыл бұрын
I think that if we all do better than Aaron, we will all be just fine, hope he’s better at customer service than testing!,
@raisethereef
5 жыл бұрын
So after watching the video and reading the comments, here is the short answer: No, you can not trust test kits or yourself. Long answer: Find a test that is consistent for YOU and stick with it. More than likely it is wrong but it doesn't really matter that it is wrong IF your tank looks good. A seasoned reefer can look at a tank and see if something isn't quite right. Doesn't matter if the alkalinity is 7 or 12 on the test kit. It could actually be 9 but if you consistently get the same "7" or "12", doesn't matter. For this reason alone, I think there are only a couple things to consider now when buying a test kit. How easy is the endpoint to read (this will mean consistency in your testing) and what is the cheapest of the kits you can properly read.
@flyingchris81
3 жыл бұрын
Usually, I really love your Videos But this one doesn’t make any sense. To use the average as the reference is completely wrong. If you don’t know the exact number of the parameter you are testing, then you can’t say anything about accuracy.
Пікірлер: 95