ICANN/DNSO
DNSO Mailling lists archives

[ga-full]


<<< Chronological Index >>>    <<< Thread Index >>>

RE: [ga] let's try to see this constructively... and not over rea ct ...


         I'm going to post one more message in this thread, and then I'll 
stop, although I have the disquieting feeling that I'm failing to 
communicate what seems to me to be a simple point.

         [1] I think that people should respond to the survey.  They should 
do so without regard to whether the survey is well- or badly-designed, 
because this is the only survey on the table, and its results will likely 
be part of the ICANN policy process.

         [2] I think that people should do whatever they can, within the 
four corners of the survey, to communicate their views, including using the 
narrative boxes to the extent they can.  (Alternatively, I know one person 
who wrote a narrative statement and sent it in *instead* of the survey.)

         [3] None of this renders it off-limits to point out that the 
survey is slanted, and the existence of narrative questions doesn't negate 
the problem.  If I were to receive a multiple-choice survey in which the 
first question asked whether the President/Prime Minister's performance was 
(a) outstanding; or (b) above average, and gave no other choices, than the 
fact that question 20 allowed me to insert narrative comments would not 
negate the bias.  Indeed, if I were to receive a multiple-choice survey in 
which a question asked whether the President/Prime Minister's performance 
was (a) outstanding; (b) above average; or (c) [fill in the blank], the 
fact that respondents could write in answers wouldn't negate the bias, 
because we'd still have the problem that the question makes it easier to 
give some answers than others.  I described one slanted question (although 
not the only one) in the whois survey earlier in this thread; anyone who 
wants to read about it can scroll down.

Jon



At 11:00 PM 6/14/2001 -0400, Marilyn Cade wrote:

>Jonathan, and Michael and others...
>
>Why not use the narrative questions to include other perspectives that you
>think aren't included, as informational points?





>-----Original Message-----
>From: Jonathan Weinberg [mailto:weinberg@mail.msen.com]
>Sent: Thursday, June 14, 2001 9:44 PM
>To: Cade,Marilyn S - LGA; 'Michael Froomkin'; 'Danny Younger'
>Cc: 'ga@dnso.org'
>Subject: Re: [ga] let's try to see this constructively... and not over
>react ...
>
>
>          I think you miss Michael's point, Marilyn.  He's not saying that
>the study is statistically invalid because it allows for narrative
>responses.  He's saying that it's invalid because it's systematically
>biased in favor of one set of results.  Thus, for example, he mentions
>Question 9, which asks whether each whois data element is "essential,"
>"desirable" or "valueless."  Problem is, almost any information has
>value.  Nobody would say that a registrant's postal address, say, is
>"valueless" -- but the survey gives no opportunity to say that it is
>nonetheless unnecessary, or that the privacy costs associated with
>including it outweigh that value.  Rather, the respondent's only other
>choices are that the information is either "essential" or
>"desirable."  That's a slanted question.
>
>          I think that everyone should fill out the survey, notwithstanding
>its bias, because -- like it or not -- the responses this survey generates
>will help shape ICANN action.  But that doesn't mean that one shouldn't
>point out the survey's rather unfortunate flaws.
>
>Jon
>
>
>
>At 09:12 PM 6/14/2001 -0400, Cade,Marilyn S - LGA wrote:
>
> >I agree with Danny's response. In my view, always easy to be critical;
> >better to be constructive.
> >
> >I regret that Michael has expressed only concerns. Isn't the survey  not to
> >determine an outcome in itself, but to try to gather information?
> >Obviously, using narrative responses is not going to result in a
> >statistically valid study; and given the method of distribution, it isn't a
> >"sample", but that wasn't and isn't the point, I would say.
> >
> >I still encourage everyone who uses WHOIS to respond and to be
>constructive.
> >
> >I believe that even sociologists sometimes do non-statistical "field
> >research"....
> >
> >
> >
> >Marilyn
> >-----Original Message-----
> >From: Michael Froomkin [mailto:froomkin@law.miami.edu]
> >Sent: Thursday, June 14, 2001 7:48 PM
> >To: Danny Younger
> >Cc: ga@dnso.org
> >Subject: Re: [ga] Re: "They're Coming To Take Me Away. Ha Ha."
> >
> >
> >I meant a professional with experience in survey design, not someone paid
>by
> >ICANN to do whatever it is ICANN pays them to do.  It goes without saying
> >that
> >no one at ICANN, or the NC, has this somewhat specialist skill (although I
> >would
> >not be surprised to find it in the GA somewhere).  No blame attaches for
> >this,
> >as there is no reason we should expect them to--it's a technical body
> >(right?),
> >not a social science research unit.  But nor should we place much credence
> >in
> >the results, whoever designed it.
> >
> >I don't see the value of a lot of energy being used to produce something
> >that
> >cannot be relied on.   The lesson is: if you are going to do a survey, run
> >it by
> >someone who designs them for a living.   Someone in a sociology department
> >or a
> >psychology department of a university who does empirical work if you can't
> >get a
> >professional pollster.
> >
> >In fact, given the bias of the design, the greater the 'outreach' for this
> >survey, the greater the damage.
> >
> >[I'll almost certainly be offline from now until Tuesday due to travel.]
> >
> >Danny Younger wrote:
> >
> > > With regard to the WHOIS survey, Michael Froomkin writes:  "I wish
>whoever
> > > wrote this had consulted a professional."
> > >
> > > Actually, we are probably quite lucky that the Names Council's WHOIS
> > > committee (of which I am also a member) wrestled this project away from
> >the
> > > ICANN staff which some time ago had formed its own WHOIS Committee
> > > http://www.icann.org/committees/whois/. Yes, it might have been more
> > > professional had the ICANN staff devoted their full-time professional
> >skills
> > > to it, but I for one am quite happy to see the uncompensated volunteer
> > > members of the Names Council showing some real initiative and a desire
>to
> > > address the problems that lay ahead.
> > >
> > > Whatever this survey may lack in "design", one has to admit there has
>been
> >a
> > > major effort made at outreach, along with a significant effort to
> >translate
> > > this survey into multiple languages (more soon to be forthcoming).
>Please
> > > bear in mind that this is only step one in a much longer process that
>will
> > > fully involve the General Assembly.    Let's not go overboard with the
> > > criticism when it is clear that we are making forward progress.
> > >
> > > --
> > > This message was passed to you via the ga-full@dnso.org list.
> > > Send mail to majordomo@dnso.org to unsubscribe
> > > ("unsubscribe ga-full" in the body of the message).
> > > Archives at http://www.dnso.org/archives.html
> >
> >--
> >This message was passed to you via the ga@dnso.org list.
> >Send mail to majordomo@dnso.org to unsubscribe
> >("unsubscribe ga" in the body of the message).
> >Archives at http://www.dnso.org/archives.html
> >--
> >This message was passed to you via the ga@dnso.org list.
> >Send mail to majordomo@dnso.org to unsubscribe
> >("unsubscribe ga" in the body of the message).
> >Archives at http://www.dnso.org/archives.html
>
>--
>This message was passed to you via the ga@dnso.org list.
>Send mail to majordomo@dnso.org to unsubscribe
>("unsubscribe ga" in the body of the message).
>Archives at http://www.dnso.org/archives.html

--
This message was passed to you via the ga-full@dnso.org list.
Send mail to majordomo@dnso.org to unsubscribe
("unsubscribe ga-full" in the body of the message).
Archives at http://www.dnso.org/archives.html



<<< Chronological Index >>>    <<< Thread Index >>>