I haven’t used them in Spark directly but here’s how they are used for computing sparse joins in a similar data processing framework:
Let’s say you want to join some data “tables” A and B. When B has many more unique keys than are present in A, computing “A inner join B” would require lots of shuffling if B, including those extra keys.
Knowing this, you can add a step before the join to compute a bloom filter of the keys in A, then apply the filter to B. Now the join from A to B-filtered only considers relevant keys from B, hopefully now with much less total computation than the original join.
Collage sounds really interesting , will check it out. Another variation on bloom filter I recently learned about is count-min-sketch. It allows for storing/incrementing a count along with each key, and can answer “probably in set with count greater than _”, “definitely not in set”.
Thanks for adding more detail on the DB use-cases!
Thanks! I’ve seen some of your great posts and LED swap mods on BLF. These are well-photographed, something which is more difficult than I realized once I tried taking some photos of lights, with phones being so good now at taking pics of people and landscapes.
I do have a ts10 in the rotation (plus two extras from when Terry from Wurkkos said the anodized colors were being discontinued) and it is as good as everyone says, although I only like it in 5700k.
The reflector of the flashlight is built so light coming from a very small source (like the filament of an incandescent bulb) is directed forward in a focused beam.
I agree, but I also think that using a modern LED with a single source of forward-facing light is fine. However the emitter would need to be properly positioned in the light.
Here’s a very similar host (from one of the best low-cost flashlight makers) showing a properly aligned LED and reflector:
I found a few references to this exact model on candlepowerforums.com which I believe has more folks who own(ed) incandescent lights. Not that has been such a long time, but LED technology advanced very quickly. Not sure if that will help your search.
Emisar D2 can be ordered with UV on the second emitter. Pictured 2nd and 4th from bottom left in OP’s photo but those all look like visible light emitters.
It’s a much smaller brand than Olight so keep that in mind if you’re looking for something with more mass market appeal.
gets hot enough to burn you if you leave it on for too long
That Fenix looks like a reliable light and is designed with temperature regulation but the limit might be pretty high and of course it is being used in an enclosure.
Sorry @xenspidey@lemmy.zip ! I did not see any because I was using an ad blocker but I didn’t intend to spam. I do think this reviewer is worth reading and (with apologies to Mr Zeroair) blocking the ads if possible.
I haven’t used them in Spark directly but here’s how they are used for computing sparse joins in a similar data processing framework:
Let’s say you want to join some data “tables” A and B. When B has many more unique keys than are present in A, computing “A inner join B” would require lots of shuffling if B, including those extra keys.
Knowing this, you can add a step before the join to compute a bloom filter of the keys in A, then apply the filter to B. Now the join from A to B-filtered only considers relevant keys from B, hopefully now with much less total computation than the original join.