Changes to Mulligan Rule

Oversoul

The Tentacled One
But your claim to evidence is a lack of blogs or mentions in tournament reports.
Nope. Didn't say that. I'm referring to the totality of the circumstances here...
  • The Magic tournament scene varies in detail across regions and locally within regions depending on tournament size, level of rules enforcement, demographics of communities, quirks of local metagames, etc.
  • To acquire good data out of such a field, a large, unbiased, preferably randomized sample would be necessary.
  • Although video-recorded matches and Magic Online matches do offer records from which data could be drawn, the broader set of tournament gameplay has no inherent record, so observers would need to go out into the field and collect samples. That is, professionals would need to be hired to carry out this data collection.
  • Observers draw attention. In matches with big-name players at large tournaments, they could blend into the crowd, but that's a small subset of tournament gameplay. For most tournaments, a stranger walking in and taking notes on random matches would lead to questioning, and this could become problematic and even dangerous. Often, nothing would come of it, but for the scope of data collection that would be necessary to get good samples, especially over many years if this is an ongoing process, something is bound to happen. If, across the board, there's only a 1% chance that players will go online and post about these strange visitors at a tournament, there'd be a massive number of reports about it. If there's only a 0.5% chance that it would lead to a hostile confrontation or involve tournament organizers or judges, that would still happen enough that we'd see plenty of reports about it. I made those numbers up, but I suspect that the actual numbers would be considerably higher.
  • Due to the way human social interactions work, the more hidden one of our hypothetical data collectors is, the more attention that will create once the individual is found out. A plain, obvious observer might get away with being generally ignored by most people, but is going to be seen by everyone and some small portion of the crowd will say something about it on the internet. A ninja hiding under tables with a periscope might be able to avoid detection sometimes, but is going to generate quite the buzz once found out.
  • In the small number of tournaments that I have attended, I've never seen mysterious note-takers randomly observing people's matches. This isn't inconsistent with a covert data collection program, as I could just always happen to miss the tournaments where it happens, but I've also seen zero reports of it happening anywhere. Zero. None. If something may or may not be happening and I'd expect that some of the people, some of the time, would talk about it, but no one is talking about it, I conclude that the thing isn't happening. This is the same approach that I take to many things in life. For example, I do not believe that the city of Tacoma, Washington is currently on fire. I cannot actually see the city of Tacoma from where I live, nor do I have a live video feed of the city. But there are a lot of other people who can see Tacoma from where they live, and if it were on fire, they would know. Only a small fraction of them would report such an event in a manner that I might see it, but because there are so very many of them, it is inevitable that in such an event, I'd see not just one report, but that some websites I frequent would be inundated with discussion and coverage of the fire. Because this is not happening, I conclude, without direct observation, that Tacoma probably isn't on fire right now.
  • Wizards of the Coast, if they wanted a tournament data collection program of the scale in question, could opt for a publicly disclosed program. This would have some advantages. Their agents would have official recognition, so they wouldn't be confronted by suspicious players or tournament organizers who wonder if these are people up to no good. They could act as a direct interface for local stores attempting to manage promotions and such, they could check up on local judges, get feedback from players who might not normally be vocal about things, run promotions themselves, find out things that they wouldn't necessarily learn if they were hiding in the shadows, etc. In contrast, I can't think of a single good reason to have a covert data collection program over an overt one.
  • Wizards of the Coast has a track record of being not merely transparent about information when it isn't a liability or spoiler, but actively sensational about it. Yeah, like any company, they have things that they don't want to disclose publicly, but that seems to be more in cases where they don't want the public to see something until the final product is out there, where they want to control the release of information so as to build up suspense as the promote a new product, or where it doesn't behoove them to air their dirty laundry. They don't go revealing what really happened behind closed doors with the shifts in attitude regarding the Reserved List. They spoil upcoming sets according to a schedule they control and when there is a leak, they investigate the source. They also do "Inside R&D" and retrospective highlights of the process behind making sets and stuff like that. I think that it'd be out of character for them to hide something so innocuous as collecting data on tournament gameplay.
  • We know a little about the sorts of people Wizards of the Coast hires. In the past, I actually applied there a few times from 2013 to 2014 (I live close WotC headquarters and I wasn't having luck with job applications for the field I got my degree in, so I was broadening my search and figured an entry-level job there might be worth going for) and still get emails about open positions (by now, I have experience with my current job and taking a pay cut to switch to a completely different industry would be a questionably decision, but I still consider it sometimes and I stay on the mailing list anyway mostly out of curiosity). But anyone who is interested can check out that stuff. We also know some of the backgrounds of the people who do work there and we know about the work people do who used to work there and have since left the company. There's definitely some diversity in people's backgrounds. I've seen computer people, artists, people who majored in communications, successful tournament players who were recruited to work on the game, games design professionals, and some other categories of professions. No career statisticians, though. I'd assume that there actually have been some, and that I just didn't see it. It seems like with how many different types of people who have worked there over the years, there's probably been a professional statistician in there somewhere. But whatever the case, it's not the norm. It's not just out of character for them to have a covert tournament gameplay data collection program. It's out of character for them to have any tournament data collection program. And in part, that's unfortunate. With this, I'm not saying that they're doing a terrible job. I bet they do have some people working on Magic who are much better with statistics than me (I took one stats course in college and applied statistics somewhat in my chemistry coursework). But it's something that I definitely think they could use more of an emphasize more. From the language they use and the decision's they do talk about, I think they're relying too much on qualitative assessment of what feels right to them and getting too much confirmation bias. This is a tangent, but there are things they could look at quantitatively and don't. Maybe they should.
  • An international tournament gameplay data collection program would be expensive. They'd have to justify such an expense. I'd imagine that this would be difficult to do.
  • Keeping data collection a secret could generate additional expenses beyond what an overt program would have. That would be really tough to justify.
  • Two people can keep a secret if three of them are dead. Magic tournaments have been popular for a long time. Unless a covert data collection program just started, there would be too many opportunities for it to be outed.
  • Wizards of the Coast has provided an alternative explanation for what information they're using to make the assessment with regard to mulligans in this case. While they could be lying to cover for their secret spy program, I see no purpose behind that, whereas their stated explanation (running tests and listening to player feedback) makes sense and seems consistent with the facts.
I can't believe that I just got suckered into seriously arguing for the non-existence of mulligan-watching spy conspiracy in the Magic tournament scene, but there you have it. Well played, Spiderman. Really though, it can be a useful exercise to articulate why a claim is ridiculous, rather than just intuiting that it is. I don't think that you really believe in the existence of this spy conspiracy fantasy yourself. You just got hung up on the idea that I couldn't debunk it. Well, I think that I now have.
 

Oversoul

The Tentacled One
Going to the extreme example, the basis for the article could be from a single tournament (let's say a Grand Prix) and from the results of the top 8 matches going forward to the finals (so that's 7 matches, right?). There were 3 mulligans taken and of those, 2 games resulted in a player just watching his opponent play and couldn't get his deck going (kind of like my Commander games with Mooseman). Since that is 66% of non-interactive games with mulligans, WOTC deemed to experiment with this new mulligan rule for Pro Tour Magic Origins.
Oh, I doubt they'd be that bad about it. I know I just accused WotC of a lack of statistical rigor where I think it would make sense for them to hire more statisticians and quantify things more, but they're still smart people. They probably wouldn't actually use just one tournament as their sole basis for a decision like this. They even came out and said that they ran tests and listened to player feedback.

Now, is one tourney sufficient enough to cause a mulligan change? Is seven matches? Is 66%?
For the numbers themselves, see p-values. As for one tournament, the problem there is a lack of randomization within a large data set. To continue with the extreme example game, what if in another tournament it was the other way around? What if the players who didn't mulligan tended to fail to interact in that tournament. Not that this would actually happen, but the point is that if you only look at one tournament, you don't know how much it might deviate from the average tournament.

You can argue "no" to one, two, any of those cases. You could argue it's arbitrary. You could argue this information should have been made public and have the players weigh in with their opinions. You could argue... anything.
I could. But I don't. Instead, I've elected to argue exactly what I've argued thus far...
  • "Interactivity" is a hazy concept and would need to be properly defined in order to be useful.
  • The goals that WotC has for what kind of interaction is desirable in a Magic game are not necessarily universal (I didn't really elaborate on this, but no one else brought it up anyway, so I'm content to leave it for now).
  • In most cases, the scry modification will not substantially influence mulligan decisions (it might change the outcome for a player who has already committed to a mulligan, but it shouldn't affect whether that player made the choice to do so in the first place).
  • Wizards of the Coast doesn't actually have a sound estimate for the percentage of Magic games that are "non-interactive" (whatever that means) due to mulligans. Because data collection that would provide this information hasn't been set up. Tournaments often keep records of wins and losses, but not mulligans or "interactivity." To get data, statistically sound data, on this would require a large program of observation and recording that does not presently exist.
  • No Spiderman, it's not that WotC has such a program and keeps it a secret because they're insane. There simply is no such program. That's silly. You're being silly.
The point is, WOTC collected *some* sort of data to make a decision.
Well, you could call just walking into the Grand Prix and shouting, "Hey guys, would you say that mulligans make for one-sided games too much?" and checking off about how many yeas you think you got compared to nays some sort of data collection. I'd call it the useless sort. I draw a distinction between some sort of data collection and good data collection.
 

Spiderman

Administrator
Staff member
<sigh> You either don't get it or are so hung up that there *has* to be some massive data collection for WOTC to make this decision that it can't be any other way. Yes, all of your bullet points make sense, if one wanted to be thorough about gathering enough data before making such a decision. But none of it matters since we don't know how it was really gathered.

Wizards of the Coast has a track record of being not merely transparent about information when it isn't a liability or spoiler, but actively sensational about it.
This in particular, can you state some examples where *actual data* was given and data collection methods explaind behind past policy/decision changes? I sure don't recall any...

I don't think that you really believe in the existence of this spy conspiracy fantasy yourself.
I never said I did, I believe you brought it up because you couldn't just believe that since WOTC's data collection methods might be different from how you imagined or would do it, that one of the only semi-possible (and thus ludicrous) reasons was that there had to be "secret observers".

Oh, I doubt they'd be that bad about it.
hence "extreme example"

They even came out and said that they ran tests and listened to player feedback.
I know, I kept mentioning that myself. But again, we don't know what those tests exactly involved (I think you said yourself it might have just been R&D running them, or it could have been in-house with the Future leagues) and who exactly were the players and how many. Keeping with the extreme example, it could have been those final 8 players.

To continue with the extreme example game, what if in another tournament it was the other way around? What if the players who didn't mulligan tended to fail to interact in that tournament.
Could have been, who said that wasn't collected either? And it turned out to be inconsequential? Again, the point is we don't know what was collected or how.

No Spiderman, it's not that WotC has such a program and keeps it a secret because they're insane. There simply is no such program. That's silly. You're being silly.
Again to repeat and clarify since this seems to be in your head, I never said there was a program. The only mention I made of "secret agents" was in response to turgy22's post and hopefully my emoji afterwards made clear of what kind of statement that was.
 

Oversoul

The Tentacled One
<sigh> You either don't get it or are so hung up that there *has* to be some massive data collection for WOTC to make this decision that it can't be any other way.
Absolutely not. They can make whatever decision they want, and I've admitted (a few times now, I think) that I believe them when they say they ran tests and listened to player feedback. That's all fine. They also made a claim about interactivity in games and have stated that they don't think that there's been enough of it. I take issue with that because they haven't defined interactivity, but this is the sort of thing they've done for a while now and that's their prerogative. Setting that aside, they connected "interactivity" to mulligans (in an inverse relationship). And that's fine too! People notice trends if they're around. WotC is bound to notice trends, and players are bound to notice trends, some of whom will talk to them about it. They can also run tests themselves, which aren't real tournament games but could reasonably be expected to simulate many aspects of those games. They could talk to judges. They could review video-recorded matches. They could see some tournament matches in person and note their own observations. I said it at some point, but of course they're not going into this blindly. While I remain skeptical of the "scry" approach to this issue (mostly because I think that it will strengthen some Legacy decks more than others and that I don't particularly want the kind of change that I think it would impel), I'm willing to admit that WotC has a lot of information to work with and that if this is the decision they make, it will have been grounded in reality. None of that is the bone of contention here.

What sparked this was that Mooseman asked if people thought that it would lower the amount of blowouts caused by bad opening hands and mulligans, then you said maybe there's data about it somewhere. I said that I didn't think that sort of data would be collected. And maybe it still hasn't been spelled out yet, but this is because "data" is quantitative. It refers to something measured. This is not the same as, say, knowledge. If you say that Wizards of the Coast has knowledge of the topic, I'd agree. But to get data, they have to record measurements. For tournament matches, this would necessitate records of what happened in those tournament matches. For a comparison between Pro Tour Magic Origins and the tournaments before it, one would need measurements from those tournaments. Not player feedback. Not checking the videos of featured matches. Not playing tests games and simulating the differences. To get data on the correlation between mulligans and any other thing, one would need to actually have records of mulligans and those other things. And that is data collection. Those other things aren't bad or wrong to have. They're just not data collection.

So I'm not saying that there has to be data collection for WotC to make a decision. I'm saying quite the opposite: they're making a decision and there won't have been data collection (or at least, not much of the particular type of data collection that I mentioned).

Yes, all of your bullet points make sense, if one wanted to be thorough about gathering enough data before making such a decision. But none of it matters since we don't know how it was really gathered.
I contend that it wasn't gathered, or rather that the information that was gathered was not data-driven.

This in particular, can you state some examples where *actual data* was given and data collection methods explaind behind past policy/decision changes? I sure don't recall any...
Eh, maybe. Since that's not a thing I talked about, I'm not sure why you'd expect me to provide it.

I never said I did, I believe you brought it up because you couldn't just believe that since WOTC's data collection methods might be different from how you imagined or would do it, that one of the only semi-possible (and thus ludicrous) reasons was that there had to be "secret observers".
So are you seriously claiming that Wizards of the Coast collects data on tournament matches, not just deck composition and wins and losses and such, but actual details about the games (like mulligans), that none of us know about this, and that somehow they also are not hiding it? Really?

I know, I kept mentioning that myself. But again, we don't know what those tests exactly involved (I think you said yourself it might have just been R&D running them, or it could have been in-house with the Future leagues) and who exactly were the players and how many. Keeping with the extreme example, it could have been those final 8 players.
Yes, but even if they kept meticulous records of every detail of those test matches, they'd still be collecting data on the test matches and not the actual tournament matches.

Could have been, who said that wasn't collected either? And it turned out to be inconsequential? Again, the point is we don't know what was collected or how.
True! We might be able to make some reasonable conclusions as to what wasn't collected, though. Take bone marrow samples, for instance. I think that if they'd collected those, I'd probably have heard about it.

Again to repeat and clarify since this seems to be in your head, I never said there was a program. The only mention I made of "secret agents" was in response to turgy22's post and hopefully my emoji afterwards made clear of what kind of statement that was.
Yeah, but to collect data on the details of tournament matches, there would necessarily have to be a program. And for a program extensive enough to get good data not to be noticed by the community, it would have to be concealed. Secret agents. I'm not saying that they're out there. I'm saying that your contention would imply them.
 

Spiderman

Administrator
Staff member
I said that I didn't think that sort of data would be collected. And maybe it still hasn't been spelled out yet, but this is because "data" is quantitative. It refers to something measured.... To get data on the correlation between mulligans and any other thing, one would need to actually have records of mulligans and those other things. And that is data collection.
And coming around again, you don't know if that is or is not actually being collected or what is behind the decision. You are using YOUR definition/idea of "data collection", using it to define what must be what WOTC has to go by to make their decision, and since you don't see any evidence of it, you are making the conclusion that there is none or no basis.

they're making a decision and there won't have been data collection (or at least, not much of the particular type of data collection that I mentioned
Continuing to beat the horse, you don't know if there is or isn't. My extreme example is an example of collecting data. Not a whole lot, but it's there.

I contend that it wasn't gathered, or rather that the information that was gathered was not data-driven.
That's fine and obviously the whole root of this division.

Eh, maybe. Since that's not a thing I talked about, I'm not sure why you'd expect me to provide it.
Because you stated
Wizards of the Coast has a track record of being not merely transparent about information...
and since "information" can mean many different things, I am narrowing down what that means to this discussion, which is data. So to put another way, I am asking you to back up your statement that WOTC has a history of providing data behind their decisions.

So are you seriously claiming that Wizards of the Coast collects data on tournament matches, not just deck composition and wins and losses and such, but actual details about the games (like mulligans), that none of us know about this, and that somehow they also are not hiding it? Really?
No. I am claiming that I don't know what kind of data WOTC collects during tournaments or any other kind of matches (I am thinking of Friday Night since that is mentioned in the article too, unless those are also considered tournaments). I *am* claiming that WOTC has kept track of mulligans-to-"noninteractive games" somehow, somewhere, somewhen. I am also claiming/saying that we don't know what the somehow, somewhere, somewhen is.

Yes, but even if they kept meticulous records of every detail of those test matches, they'd still be collecting data on the test matches and not the actual tournament matches.
Why does that matter, whether it's test matches or tournament matches? It's data.

We might be able to make some reasonable conclusions as to what wasn't collected, though.
Yes, you can. Which doesn't matter because what may seem reasonable doesn't mean it actually happened.
Yeah, but to collect data on the details of tournament matches, there would necessarily have to be a program....
Still beating the horse, you are making the premise that there *has* to have been multiple tournament matches involved and going from there. And you don't know.
 
R

rokapoke

Guest
While you two are fighting about, essentially, semantics, I'll add my two cents on the whole issue. Nothing like an unsolicited opinion, eh?

WotC has historically done "extensive" internal testing of new sets, to identify potentially broken cards, etc. I personally feel (opinion, not fact!) that this decision will be like some of those -- meaning that it won't solve all the problems, and in fact may cause new ones (like Oversoul's mentioned potential strengthening of certain Legacy decks). Remember how artifact lands were considered overpowered during Mirrodin block because somebody discovered an interaction that wasn't "broken" during WotC playtesting?

Anyway, I don't care about the change, because I'm not a tournament player. I've never been anywhere close to that good. I'm just saying that sometimes WotC knows what's best for the game, and sometimes they don't. And this move can fit in either camp; time will tell.
 

Oversoul

The Tentacled One
And coming around again, you don't know if that is or is not actually being collected or what is behind the decision.
I've spent a fair bit of text explaining why I don't believe that this sort of data collection is going on. You've simply repeated that I don't know, without justifying that position in any way. Also, the article that Melkor linked to included a brief summary of what is behind the decision, at least according to Helene Bergeot, who seems to be writing on behalf of the company as a whole to explain this. I'm merely taking WotC at their word here that what they say is behind the decision is, in fact, behind the decision. You've decided that I don't know anyway. Well, I think I do, so I guess we're "coming around again."

You are using YOUR definition/idea of "data collection", using it to define what must be what WOTC has to go by to make their decision, and since you don't see any evidence of it, you are making the conclusion that there is none or no basis.
So firstly, and most importantly, you already accused me of insisting that WotC make decisions based on some standard of mine and I repudiated that. Could you please cut it out with the strawman?

Secondly, I didn't make up my own definition of data collection. I was referring to what data collection is. If you look it up in a stats textbook or even a general encyclopedia, I'm confident that it would be consistent with what I said and I'd go further and say that it would probably have the word "measure" or "measurement" in there.

Thirdly, I've made a contention that a total lack of evidence for something in an instance where evidence could reasonably expect to be found constitutes evidence against the proposition. I realize that "absence of evidence is not evidence of absence" is a popular aphorism, but I totally disagree with it and I've explained why. You are free to, in turn, disagree with me, but wildly accusing me of saying things that I can easily look back and see I haven't said isn't a very good rebuttal.

Continuing to beat the horse, you don't know if there is or isn't. My extreme example is an example of collecting data. Not a whole lot, but it's there.
If you say it enough times, maybe I'll start believing it...

Because you stated and since "information" can mean many different things, I am narrowing down what that means to this discussion, which is data. So to put another way, I am asking you to back up your statement that WOTC has a history of providing data behind their decisions.
You know, if you'd just come out and asked about what kind of instances I had in mind when I said that WotC had a track record of transparency allowing for the obvious sorts of exceptions, I think I'd probably have looked for the examples that I could remember off the top of my head. I don't have any idea whether you'd agree with me, but it's an interesting topic anyway and I sometimes like revisiting old things and seeing how they compare to my memories of them. But you took my statement about information, revised it to be about data, and then challenged me to back up my claim. Hey, if I'd meant "data" there, I'd have used the word. And being challenged to back up your goalpost-shifted version of my statement, all after you've failed to back up anything at all? Well, that particular sort of investigation just lost its appeal.

No. I am claiming that I don't know what kind of data WOTC collects during tournaments or any other kind of matches (I am thinking of Friday Night since that is mentioned in the article too, unless those are also considered tournaments).
Yes, FNM is traditionally run as a tournament. Man, are you ever dating yourself...

I *am* claiming that WOTC has kept track of mulligans-to-"noninteractive games" somehow, somewhere, somewhen. I am also claiming/saying that we don't know what the somehow, somewhere, somewhen is.
Hey, read the article. She mentioned that they ran multiple tests. I mean, if you are inclined to believe that WotC just outright lies about everything, then that's another matter.

Why does that matter, whether it's test matches or tournament matches? It's data.
You want me to explain to you, a grown adult, why test matches are different from tournament matches?

Yes, you can. Which doesn't matter because what may seem reasonable doesn't mean it actually happened.
Wait, what? Really? I mean, if you comprehended my "bone marrow sample" quip and are still on your "you can't really know" broken record then yeah, I mean, I'm just not willing to be as agnostic as you apparently are about things and I guess that's that...

Still beating the horse, you are making the premise that there *has* to have been multiple tournament matches involved and going from there. And you don't know.
I see that you like telling me that I don't know a lot. What specific thing is it that you are stating it is that I do not know?
 

Oversoul

The Tentacled One
WotC has historically done "extensive" internal testing of new sets, to identify potentially broken cards, etc. I personally feel (opinion, not fact!) that this decision will be like some of those -- meaning that it won't solve all the problems, and in fact may cause new ones (like Oversoul's mentioned potential strengthening of certain Legacy decks). Remember how artifact lands were considered overpowered during Mirrodin block because somebody discovered an interaction that wasn't "broken" during WotC playtesting?

Anyway, I don't care about the change, because I'm not a tournament player. I've never been anywhere close to that good. I'm just saying that sometimes WotC knows what's best for the game, and sometimes they don't. And this move can fit in either camp; time will tell.
I agree. To be fair, WotC doesn't really bother with Legacy anymore anyway and if this type of mulligan becomes the new norm, it should probably be judged more for its effects in Standard and Modern (and maybe Limited formats), since that's what they had in mind.

In their podcast about Vintage, Kevin Cron and Stephen Menendian discussed how Vintage, because of the nature of the decks in it, clearly demonstrates how the scry mulligan benefits some decks more than it does others. The most extreme case right now is Dredge. Vintage Dredge decks really, really want Bazaar of Baghdad in their opening hand, so much so that they include Serum Powder to help mulligan into it. Having that extra scry gives them just one more chance to find it on the first turn if things go wrong, and that's the difference between doing nothing and going off. A deck that is more consistent and doesn't need particular cards in its opening hand isn't hurt by the new mulligan, but doesn't stand to potentially gain as much.
 

Spiderman

Administrator
Staff member
rokapoke said:
Nothing like an unsolicited opinion, eh?
Naw; rather, I think it's about time others stepped in :)

rokapoke said:
I personally feel (opinion, not fact!) that this decision will be like some of those -- meaning that it won't solve all the problems...Anyway, I don't care about the change, because I'm not a tournament player.
To be clear, this isn't an overall "game change", but rather one affecting just one tournament. I see it as another test collecting more data.

turgy22 said:
If I had a hundred thousand dollars...
Not just settling for nickels, huh? :D

Oversoul said:
I've spent a fair bit of text explaining why I don't believe that this sort of data collection is going on. You've simply repeated that I don't know, without justifying that position in any way.
And I agreed that it's fine that you have your opinion on the why. But I have justified my "You don't know" simply because you haven't shown that you do by putting forth any real facts, not opinions. When you show me those, then my position is unjustified.

Oversoul said:
So firstly, and most importantly, you already accused me of insisting that WotC make decisions based on some standard of mine and I repudiated that.
I'm not going to "cut it out" because you are not getting it. Giving me a standard/general/accepted definition of "data collection" is great. Wonderful. Thanks very much. What you are NOT showing me is where it says that WOTC uses this standard to make their decision. Is there a statement on WOTC's website that says "Before making any decision, we collect data using the "standard accepted practice" of data collection"? Did an employee post something to that effect in one of their articles? I don't think so, and until you give that "link" (web or proof), I'm saying you can't tie whatever the standard of data collection is to what WOTC just *absolutely has to do" before making policy decisions.

Oversoul said:
You know, if you'd just come out and asked about what kind of instances I had in mind when I said that WotC had a track record of transparency allowing for the obvious sorts of exceptions... But you took my statement about information, revised it to be about data, and then challenged me to back up my claim. Hey, if I'd meant "data" there, I'd have used the word.
I had to revise it because as I said before, "information" does not necessarily mean " data", which is what we're specifically talking about in this point. Now that I explicitly know you didn't mean "data" in the first place, that whole bullet point means absolutely nothing to me, since you could have provided whatever "information" that would have probably had nothing to do with this whole data collection argument. Which I don't think you could have provided anyway and so all of that was an empty point anyway.

Oversoul said:
Hey, read the article. She mentioned that they ran multiple tests. I mean, if you are inclined to believe that WotC just outright lies about everything, then that's another matter.
That does not contradict my statement that you quoted, so the fact that you responded to it means you don't understand what I said.

Oversoul said:
You want me to explain to you, a grown adult, why test matches are different from tournament matches?
That's what I said.

Oversoul said:
I see that you like telling me that I don't know a lot.
About this? Yes. Because you haven't shown me that you do.

Oversoul said:
What specific thing is it that you are stating it is that I do not know?
You do not know the method of the data collection that determined WOTC's decision to implement the mulligan rule change for Pro Tour Magic Origins.* And when I say method, I mean the number of matches, where the matches took place (including environment so by extension who played those matches), the period of time collecting the data about matches, and what is meant by noninteractive to make that a data point. And since you do not know what data was collected or how, making the statement that it could not absolutely have occurred during tournaments where players were present is erroneous.

* I assume we both agree that some sort of data was collected, since I already believe it and you have mentioned the "test" line in your previous posts and that you're willing to take WOTC's word about it. If this assumption is incorrect and you don't believe any data was collected, then the specific thing that I am stating that you do not know can be revised to "You do not know that data wasn't collected, period" since the method or the how/why/when etc doesn't even matter then.
 

Spiderman

Administrator
Staff member
I also want to be clear in that you are free (as always) to say what you want, draw your conclusions, etc. I am NOT saying that because I think you are wrong, that you can't say it.

It is clear that I will never convince you to change your opinion nor you convince me to change mine (as of now). I think you are wrong, you think I am wrong, and that's pretty much where it's going to stay. So unless some new information from another source comes to light, I don't think there's any further use in discussing further.
 

Spiderman

Administrator
Staff member
Me said:
I assume we both agree that some sort of data was collected, since I already believe it and you have mentioned the "test" line in your previous posts and that you're willing to take WOTC's word about it. If this assumption is incorrect and you don't believe any data was collected, then the specific thing that I am stating that you do not know can be revised to "You do not know that data wasn't collected, period" since the method or the how/why/when etc doesn't even matter then.
Actually, looking back on the first page of the thread, I'm not even sure what you are stating or what your position is now. As you alluded to in a more recent post, this all stemmed from a question from Mooseman to which I answered/asked that WOTC should have have data, implicitly referring to Pro Tour Magic Origins since the mulligan changes were for that specific tournament and it seems obvious (!) that WOTC would keep track of mulligans-to-noninteractive games for it. Somehow in your next reply/post, you seemingly broadened the idea of data capturing to any past tournament/event leading up to Pro Tour Magic Origins and gave your reasons/thoughts about why that couldn't have possibly happened, etc. So even stemming from there, it seems the question of whether data collection happened solely during Pro Tour Magic Origins and if that could be made available morphed into whether data collection happened at all in the past.

So just from that, I took the position that yes, "data collection" happened and you took the position that it didn't (and frankly, it looks like those mere two words triggered something in you). Now in your more recent posts you have cited the "WOTC performed tests" line from the article and that you are taking WOTC at their word, which seems to mean that you are accepting of the idea that tests were done (and to me, that means data was collected) but not to "standard data collection methods". So to me, *you* shifted the goalposts.

So perhaps maybe you'd better state what you think this is all about.
 

Spiderman

Administrator
Staff member
I know! :( Perhaps once I get clarification on what he thinks this is about that further discussion is useless ;)
 

Oversoul

The Tentacled One
And I agreed that it's fine that you have your opinion on the why. But I have justified my "You don't know" simply because you haven't shown that you do by putting forth any real facts, not opinions. When you show me those, then my position is unjustified.
So, for example, the part about how Wizards of the Coast has an article up referring to tests and to player feedback, but not to data collection of game details from tournaments, is not a fact, but merely my opinion? Really?

I'm not going to "cut it out" because you are not getting it.
Oh. Well, in that case, perhaps we should discuss your claim that guano tastes great or your claim that the moon is made of green cheese and that we should enslave the Canadians and force them to mine it for us. That sounds way more entertaining than a hypothetical mulligan-recording arm of WotC.

Giving me a standard/general/accepted definition of "data collection" is great. Wonderful. Thanks very much. What you are NOT showing me is where it says that WOTC uses this standard to make their decision.
Probably because I have no indication that they've done so.

Is there a statement on WOTC's website that says "Before making any decision, we collect data using the "standard accepted practice" of data collection"? Did an employee post something to that effect in one of their articles? I don't think so, and until you give that "link" (web or proof), I'm saying you can't tie whatever the standard of data collection is to what WOTC just *absolutely has to do" before making policy decisions.
Like I said, I have no indication that Wizards of the Coast ever so much as hinted at data collection being a component of the scy mulligan thing. As far as I can tell, that's something you made up on your own. And the standard practices involved in data collection are standard for good reasons. They prevent various types of bias from skewing results. They're widely accepted across every field involving data collection. You have some goal in mind contriving a scenario in which this one company deliberately flaunts agreed-upon best practices? You have some evidence for the claim in the first place?

I had to revise it because as I said before, "information" does not necessarily mean " data", which is what we're specifically talking about in this point. Now that I explicitly know you didn't mean "data" in the first place, that whole bullet point means absolutely nothing to me, since you could have provided whatever "information" that would have probably had nothing to do with this whole data collection argument. Which I don't think you could have provided anyway and so all of that was an empty point anyway.
I'll reiterate the reason I mentioned transparency, since you still seem to think that it means "nothing" if it isn't about data-sharing.
  • I made a list of circumstances that I think, taken together, support two propositions. The first proposition is that if Wizards of the Coast were to embark on an overt program of data collection of details from tournament matches, the community would be aware of it. The second proposition is that a covert program of such data collection would be implausibly difficult to hide, and that going to such lengths to hide it wouldn't yield any benefit I can think of.
  • The company's track record could potentially support either proposition. When they engage with the community and reveal some details about the inner workings of the company, it makes the idea of such a covert program seem out of character. And for an overt program, I'd expect that they'd mention it at some point, which they haven't.
  • Neither of these is foolproof, and I'll admit that right now. They could be self-promoting and sharing information as a smokescreen to cover up things that they want to hide. They could have reason to want to hide a data collection program of this sort, and perhaps I just can't think of it. But I find those things unlikely.
That does not contradict my statement that you quoted, so the fact that you responded to it means you don't understand what I said.
For reference...

"I am also claiming/saying that we don't know what the somehow, somewhere, somewhen is."

How: With Magic cards. That's how one tests for issues regarding Magic cards.
Where: Renton, Washington. That's where she works.
When: Well, it's got to be some time recent enough to be relevant.

Of course, I don't know whether the cards were physical or digital versions. I don't know what kind of sleeves they might have had or how many trials were performed. I don't know what decklists they used. But, if I take her at her word (which I have no reason not to), I'd posit that it's Magic cards being used in this testing. And while I can't say exactly where or when, I don't see why it matters.

That's what I said.
Well, tournaments are participated in by competitive players vying for glory and prizes. This results in its own environment that is not replicated in other Magic games. Tests are often performed with the some of the same decks that are used in tournaments, but because they are not performed under the same circumstances, tests are not a perfect simulation for an actual tournament. Data drawn from test matches may not show the same patterns of results as data drawn directly from tournament matches. To ascertain the extent to which test data might differ from tournament data, it is necessary to collect both kinds of data for comparison.

About this? Yes. Because you haven't shown me that you do.
About the thing you made up in your own head? Yeah, I'm afraid my knowledge is a little shaky on that one...

You do not know the method of the data collection that determined WOTC's decision to implement the mulligan rule change for Pro Tour Magic Origins.* And when I say method, I mean the number of matches, where the matches took place (including environment so by extension who played those matches), the period of time collecting the data about matches, and what is meant by noninteractive to make that a data point.
Hold on. One of the first things that I said in this thread was that interactivity had not actually been defined. I went on to, somewhat cynically although not unjustly so, expound on how interaction can take different forms in Magic games and that I think WotC is prioritizing some interaction and overlooking other interaction. But the crux of this was that it doesn't make sense to talk about the frequency of non-interactive matches without qualifying what the characteristics of an interactive match are. The conversation steered away from that topic, but it is a thing that I said. Now you want to use it against me? Like, you're proposing that WotC has a secret definition of interactivity, and that because I lack access to it, I can't make an assessment? Are we really going back to the conspiracy stuff here?

And since you do not know what data was collected or how, making the statement that it could not absolutely have occurred during tournaments where players were present is erroneous.
I don't have to know where and how data was collected to know where and how it wasn't. For example, it couldn't have been on my bed while I was sleeping in it, as that would have woken me up. It couldn't have been on the surface of the sun, because it's too hot there. It couldn't have been throughout international Magic tournaments, because that would draw the attention of the community and they'd talk about it. They didn't, so it didn't happen that way.

I'm not saying that no data has ever been collected at any tournament. In fact, every feature match provides a record of data that WotC (or anyone) could use to generate statistics. But for reasons I've already mentioned, this could not yield a good data set. And I'm not even saying that if WotC implements the new rule that their reasoning will be bad or unjustified, but rather that they will not have taken a data-driven approach to the issue. It's entirely possible that they wouldn't need to! Here's a hypothetical scenario...
  1. Wizards of the Coast gets a lot of player feedback that mulligans often lead to non-interactive games.
  2. Wizards of the Coast observes that they think this is happening too. They examine the problem and someone gets the idea to implement a "scry 1" as part of the mulligan process.
  3. Wizards of the Coast employees play games against each other with the new rule and the games seem to be more interactive.
  4. Wizards of the Coast tests the new rule at PT Magic Origins. The new mulligan gets a lot of positive feedback and doesn't seem to cause any problems. Observation seems to reveal fewer non-interactive games.
  5. Wizards of the Coast implements the new version of the mulligan.
At no point does anything get measured here. WotC isn't using tables of results compiled from records of matches to make their decision in this scenario.
 

Oversoul

The Tentacled One
It is clear that I will never convince you to change your opinion nor you convince me to change mine (as of now).
Well, if you go into things with your mind made up that it won't be changed, you'll probably get what you expect.

I think you are wrong, you think I am wrong, and that's pretty much where it's going to stay. So unless some new information from another source comes to light, I don't think there's any further use in discussing further.
I think that you're not even wrong.

Actually, looking back on the first page of the thread, I'm not even sure what you are stating or what your position is now. As you alluded to in a more recent post, this all stemmed from a question from Mooseman to which I answered/asked that WOTC should have have data, implicitly referring to Pro Tour Magic Origins since the mulligan changes were for that specific tournament and it seems obvious (!) that WOTC would keep track of mulligans-to-noninteractive games for it. Somehow in your next reply/post, you seemingly broadened the idea of data capturing to any past tournament/event leading up to Pro Tour Magic Origins and gave your reasons/thoughts about why that couldn't have possibly happened, etc. So even stemming from there, it seems the question of whether data collection happened solely during Pro Tour Magic Origins and if that could be made available morphed into whether data collection happened at all in the past.
Lacking time travel, I can't say whether WotC will have observers recording data on mulligans at PT Magic Origins. Also, lacking time travel, WotC would not yet have data regarding mulligans at PT Magic Origins, although they could choose to collect it.

If the idea is to find out whether the scry mulligan has an effect on "interactivity" in games, only having data from a tournament with it and not any data from tournaments without it doesn't really seem helpful.

So just from that, I took the position that yes, "data collection" happened and you took the position that it didn't (and frankly, it looks like those mere two words triggered something in you).
Triggered?

Now in your more recent posts you have cited the "WOTC performed tests" line from the article and that you are taking WOTC at their word, which seems to mean that you are accepting of the idea that tests were done (and to me, that means data was collected)
But tests being done and data being collected aren't actually the same thing...

but not to "standard data collection methods". So to me, *you* shifted the goalposts.
I'm not even taking a shot at your goal. At any point, I could challenge you to demonstrate your purported data collection. You haven't provided a shred of evidence for it and I don't believe that you could. Instead, I took the chance to explain why I think that, given the circumstances, it is reasonable for me to say that I know that there isn't a data collection program on mulligan results in Magic tournaments. This is contingent on a lot of nuances of the situation. I don't know how good of a case I really made, since that's dependent on the reader and no on else is commenting on it. But the fact remains that this entire case is superfluous: a mental exercise that I chose to undertake for the fun of it. Because if you want to claim that there really is a program in place to collect data of this sort, the onus is on you to support that claim, not on me to negate it. At no point have I forgotten this. I just think that it's more interesting to go to the trouble of negating the claim anyway.

And no, "data collection" of in-house simulations isn't the same as actually having data on mulligans in real tournaments. I should think that it would be obvious. I mean, you led with your whole "WotC has the data." If it's data from games that they played against each other, then that's no different from me playing some games with a friend, recording them, and then calling it data. The whole point of data is that it is representative. I could take a random number generator and use that to make "data." It's numbers! But it's not data of the thing in question. And the thing in question, as far as I can tell, is tournament Magic games.

So perhaps maybe you'd better state what you think this is all about.
Well, you said that there might be data out there somewhere about the amount of blowouts caused by bad opening hand draws and mulligans. I said that it was too subjective (blowouts aren't a well-defined thing, so measuring them isn't feasible) and that collecting data on mulligan frequency would require a change to the way tournaments are run.

In most current tournaments, players are not constantly monitored and only report the win/loss results of the games they've played. So tournament organizers have data (some of which gets published) as to which decks won which games when. But no one is watching and recording the number of mulligans or the other game details. Tournaments could have observers at every table to note the details of the games themselves, but they don't.

You contended that WotC has data on this, even if the players don't. I disagreed, and I noted from the outset that this was contingent on a rather strict notion of what would constitute viable data for tournaments played around the world in a card game.

I think that part of the breakdown here is that because you've now taken the stance that data collection must be happening, it must be the case that in-house test games are a valid source for data collection and that WotC has actually collected data on these test games.
 

Spiderman

Administrator
Staff member
Okay, I see that there's no hope of continuing the discussion further. You are hung up on the words "data collection" and what that strictly means and that since there's apparently no way WOTC could have adhered to that, you're making the conclusion that it could never have happened. You are also hung up on since WOTC hasn't explained what they mean by interactivity, no data can ever be collected that's "objective" as you see it and understand.

You also still aren't understanding what I'm saying as evidenced by your replies and what you keeping quoting me on.

So that's that.
 

Oversoul

The Tentacled One
Okay, I see that there's no hope of continuing the discussion further. You are hung up on the words "data collection" and what that strictly means and that since there's apparently no way WOTC could have adhered to that, you're making the conclusion that it could never have happened.
No one was talking about data collection with regard to this change until you made it up on your own. It is entirely your own idea, and yet I'm the one who is hung up on it?

The best hope for "continuing the discussion further" is that it presents an opportunity for you to attribute a position to me that is actually my position—in other words, to stop strawmanning me. Like my "conclusion that it could never have happened." Where'd that come from? Not from anything that I've actually said.

You are also hung up on since WOTC hasn't explained what they mean by interactivity, no data can ever be collected that's "objective" as you see it and understand.
Another straw man. Do you think that at some point I will suddenly stop being able to tell the difference between the things that I've actually said and the things that you've said and then attributed to me? Or can you really not tell the difference yourself?

You also still aren't understanding what I'm saying as evidenced by your replies and what you keeping quoting me on.
Oh, it must be some reading error on my part! I mean, it looks like you keep saying that I've said something and that the things I've actually said don't correspond to the thing that you've said I've said. But really, some other, different thing is happening and I just don't get it. Right...
 
Top