Show counts of intuition misses #7

some2112 · 2023-11-18T08:23:14Z

Can you add a mini Hawk Eye with features different from the YZY version?

I hope you can provide some source code references.

The first point is to only calculate the matching when the AI's recommended move policy is greater than 0.7. The purpose is to see the matching in situations where the AI's recommendation is very certain.

The second point is that the calculation condition for global matching is that as long as a player's move matches the AI's policy with a probability of 0.1% or more, it is considered a match. This way, beginners can better understand if their moves align with the AI's recommendations.

Please display this information directly on the main screen like this

.

kaorahi · 2023-11-18T14:33:58Z

Are you suggesting a kind of performance report through a game like sanderland/katrain#388 ?

Unfortunately, this repository is currently in a "maintenance phase", and we are not focusing on adding new features in principle. In my impression, implementing this would require a certain amount of effort, and its practical usefulness isn't entirely clear. For example, the policy of the best move can be less than 0.7 after a long search. Do we count this as a "bad" move? What if all the policies are less than 0.1? The threshold 0.1 might be too large since the policy of "komoku" as the first move is only 0.026.

Anyway, the following tasks would be necessary for this feature:

Add policy and max_policy to BoardData.java to record them.
Count "bad" moves and show the results in both BasicInfoPane.java and LizzieFrame.java.

I'm afraid that this feature might not be very convenient without an additional functionality to jump to the corresponding moves, which would involve extra work.

kaorahi · 2023-11-18T14:39:56Z

As a quick experiment, I implemented a slightly modified version on lizgoban lowpolicy_231118a branch because it is much easier to me. Gray squares are drawn on stones if "policy < 0.01" (thin) or "max_policy > 0.7 && policy < 0.1" (thick). The former is somewhat distracting to me but the latter is interesting. I'll try it for a while to test its usefulness.

some2112 · 2023-11-18T15:14:28Z

maybe i show some examples

The reason why we only focus on bestmove+high policy is because these moves represent the mastery of the basic skills of Go.

When someone behaves like this, it's incorrect.

and like this

Improving the hit rate of this bestmove+high policy is very important for beginners

some2112 · 2023-11-18T15:21:59Z

When we use yzy version

With the emergence of too many bestmove+high policies, the accuracy in the yzy version will be particularly affected by the actual gameplay. Some simple long formulas will increase the degree of agreement.
At the same time, the accuracy of each set cannot reflect progress and regression.
But if we only count the hit rate of bestmove+high policy, it can reflect whether a go player has made progress.

kaorahi · 2023-11-20T11:21:24Z

I agree that the concept of "overlooked high-policy moves" is an interesting focus. I'm currently testing it on lizgoban to get a feel for it. So far, it seems to give learners some hints about priority among various mistakes in a game. But its implementation on lizzie is not a very light task for me and I cannot make a promise on the development, unfortunately. I haven't used lizzie personally for a long time...

Also, I have some concerns:

(Feature) Just showing counts is useful enough?
(Presentation) Most users might wonder, "What are these numbers?"

some2112 · 2023-11-21T04:22:46Z

I think showing counts and percentages is enough

Maybe you can add 2 titles:

Basic skill matching rate 基本功吻合率

serious error rate 嚴重失誤率

kaorahi · 2023-11-23T21:19:41Z

wording idea: "Intuition Misses 2/5 40% (minor 111/250 44%)"

some2112 · 2023-11-24T05:57:38Z

NICE !

When will the source code be available?

kaorahi · 2023-11-25T10:42:20Z

Experimental implementation. Untested at all!

https://github.com/kaorahi/lizzie/tree/kaorahi7_policymiss

To display the counts of intuition misses, edit your config.txt like this. Insert several lines into the existing "ui" section:

{
  ...,
  "ui": {
    "experimental-intuition-miss": {
      "show-intuition-miss": true,
      "bestmove-policy-threshold": 70,
      "played-policy-threshold": 10,
      "minor-policy-threshold": 10
    },
    ...
  }
}

Thresholds (%) are used as follows.

Intuition Miss: bestmovePolicy > bestmove-policy-threshold && policy < played-policy-threshold
minor: policy < minor-policy-threshold

Please don't have high expectations. Lizzie is not my main project, and I actually won't be using it personally.

some2112 · 2023-11-26T09:04:20Z

It worked.
Your help is very useful, thank you very much.

some2112 · 2023-11-27T11:30:15Z

If I want to start statistics after move 6, how should I modify the source code?

kaorahi · 2023-11-27T13:57:11Z

c768458 with config.txt:

{
  ...,
  "ui": {
    "experimental-intuition-miss": {
      "show-intuition-miss": true,
      "skip-first-moves": 6,
      "bestmove-policy-threshold": 70,
      "played-policy-threshold": 10,
      "minor-policy-threshold": 10
    },
    ...
  }
}

If you want to help with the development, I would be happy to hear what you think after you test it for a month or so.

After actual testing, do you still think this is useful?
Any ideas on practical default values for these parameters?

some2112 · 2023-12-01T12:32:18Z

String[] msgs = Lizzie.frame.intuitionMissMessages();
if (msgs != null && Lizzie.board.getHistory().getCurrentHistoryNode().isMainTrunk()) {
setPanelFont(g, (float) (height * 0.08));
int w0 = g.getFontMetrics().stringWidth(msgs[0]);
int w1 = g.getFontMetrics().stringWidth(msgs[1]);
int w2 = g.getFontMetrics().stringWidth(msgs[2]);
int w3 = g.getFontMetrics().stringWidth(msgs[3]);
int w4 = g.getFontMetrics().stringWidth(msgs[4]);
int w5 = g.getFontMetrics().stringWidth(msgs[5]);
int w6 = g.getFontMetrics().stringWidth(msgs[6]);
int w7 = g.getFontMetrics().stringWidth(msgs[7]);
int w8 = g.getFontMetrics().stringWidth(msgs[8]);
int w9 = g.getFontMetrics().stringWidth(msgs[9]);
int w10 = g.getFontMetrics().stringWidth(msgs[10]);
int w11 = g.getFontMetrics().stringWidth(msgs[11]);
int w12 = g.getFontMetrics().stringWidth(msgs[12]);
int w13 = g.getFontMetrics().stringWidth(msgs[13]);
int w14 = g.getFontMetrics().stringWidth(msgs[14]);
g.setColor(Color.BLACK);
g.drawString(msgs[0], posX + width / 6 - w0 / 2, posY + height * 3 / 10);
g.drawString(msgs[1], posX + width * 7 / 20 - w1 / 2, posY + height * 3 / 10);
g.setColor(Color.RED);
g.drawString(msgs[2], posX + width / 2 - w2 / 2, posY + height * 3 / 10);
g.setColor(Color.WHITE);
g.drawString(msgs[3], posX + width * 13 / 20 - w3 / 2, posY + height * 3 / 10);
g.drawString(msgs[4], posX + width * 5 / 6 - w4 / 2, posY + height * 3 / 10);
g.setColor(Color.BLACK);
g.drawString(msgs[5], posX + width / 6 - w5 / 2, posY + height * 2 / 5);
g.drawString(msgs[6], posX + width * 7 / 20 - w6 / 2, posY + height * 2 / 5);
g.setColor(Color.RED);
g.drawString(msgs[7], posX + width / 2 - w7 / 2, posY + height * 2 / 5);
g.setColor(Color.WHITE);
g.drawString(msgs[8], posX + width * 13 / 20 - w8 / 2, posY + height * 2 / 5);
g.drawString(msgs[9], posX + width * 5 / 6 - w9 / 2, posY + height * 2 / 5);
g.setColor(Color.BLACK);
g.drawString(msgs[10], posX + width / 6 - w10 / 2, posY + height / 2);
g.drawString(msgs[11], posX + width * 7 / 20 - w11 / 2, posY + height / 2);
g.setColor(Color.RED);
g.drawString(msgs[12], posX + width / 2 - w12 / 2, posY + height / 2);
g.setColor(Color.WHITE);
g.drawString(msgs[13], posX + width * 13 / 20 - w13 / 2, posY + height / 2);
g.drawString(msgs[14], posX + width * 5 / 6 - w14 / 2, posY + height / 2);
setPanelFont(g, (float) (height * 0.1));
int w15 = g.getFontMetrics().stringWidth(msgs[15]);
int w16 = g.getFontMetrics().stringWidth(msgs[16]);
FontMetrics rating = g.getFontMetrics();
int textHeight = rating.getHeight();
int textAscent = rating.getAscent();
int textPosY = posY + height / 2 + (height / 4 - textHeight) / 2 + textAscent;
if (showrating > 42) {
g.setColor(Color.BLACK);
g.drawString(msgs[15], posX + width / 7 - w15 / 2, textPosY);
g.setColor(Color.WHITE);
g.drawString(msgs[16], posX + width / 7 * 6 - w16 / 2, textPosY);
}

  int bpol = numMajorIntuitions[0] - numMajorMisses[0];
int bpolt = numMajorIntuitions[0];
int wpol = numMajorIntuitions[1] - numMajorMisses[1];
int wpolt = numMajorIntuitions[1];
int bcal = numBestMoveMatches[0] + numMajorMisses[0] - numMajorIntuitions[0];
int bcalt = numMoves[0] - numMajorIntuitions[0];
int wcal = numBestMoveMatches[1] + numMajorMisses[1] - numMajorIntuitions[1];
int wcalt = numMoves[1] - numMajorIntuitions[1];
int bdir = numMoves[0] - numMinorMisses[0];
int bdirt = numMoves[0];
int wdir = numMoves[1] - numMinorMisses[1];
int wdirt = numMoves[1];
showrating = numMoves[0] + numMoves[1];
double blackelo = 0.0;
double whiteelo = 0.0;
if ((bpolt * 1.1 < wpolt) && (bpol < wpol)) {
  blackelo -= ((wpol - bpol) * 30);
}
if ((wpolt * 1.1 < bpolt) && (wpol < bpol)) {
  whiteelo -= ((bpol - wpol) * 30);
}
if (bpolt > 0) {
  blackelo += (bpol * 3200 / bpolt);
}
if (wpolt > 0) {
  whiteelo += (wpol * 3200 / wpolt);
}
if (bcalt > 0) {
  blackelo += (bcal * 4800 / bcalt);
}
if (wcalt > 0) {
  whiteelo += (wcal * 4800 / wcalt);
}
if (bdirt > 0) {
  blackelo += (bdir * 5000 / bdirt);
}
if (wdirt > 0) {
  whiteelo += (wdir * 5000 / wdirt);
}
String brating = getGoRating(blackelo);
String wrating = getGoRating(whiteelo);
String[] ret = {
  String.format("%d/%d", bpol, bpolt),
  String.format("%s", percentString(bpol, bpolt)),
  String.format("(POL)"),
  String.format("%s", percentString(wpol, wpolt)),
  String.format("%d/%d", wpol, wpolt),
  //
  String.format("%d/%d", bcal, bcalt),
  String.format("%s", percentString(bcal, bcalt)),
  String.format("(CAL)"),
  String.format("%s", percentString(wcal, wcalt)),
  String.format("%d/%d", wcal, wcalt),
  //
  String.format("%d/%d", bdir, bdirt),
  String.format("%s", percentString(bdir, bdirt)),
  String.format("(DIR)"),
  String.format("%s", percentString(wdir, wdirt)),
  String.format("%d/%d", wdir, wdirt),
  String.format("%s", brating),
  String.format("%s", wrating),
};
return ret;

}

private static String getGoRating(double elo) {
if (elo <= 2100) {
return "30k";
} else if (elo > 2100 && elo <= 2250) {
return "29k";
} else if (elo > 2250 && elo <= 240) {
return "28k";
} else if (elo > 2400 && elo <= 2550) {
return "27k";
} else if (elo > 2550 && elo <= 2700) {
return "26k";
} else if (elo > 2700 && elo <= 2850) {
return "25k";
} else if (elo > 2850 && elo <= 3000) {
return "24k";
} else if (elo > 3000 && elo <= 3150) {
return "23k";
} else if (elo > 3150 && elo <= 3300) {
return "22k";
} else if (elo > 3300 && elo <= 3450) {
return "21k";
} else if (elo > 3450 && elo <= 3600) {
return "20k";
} else if (elo > 3600 && elo <= 3750) {
return "19k";
} else if (elo > 3750 && elo <= 3900) {
return "18k";
} else if (elo > 3900 && elo <= 4050) {
return "17k";
} else if (elo > 4050 && elo <= 4200) {
return "16k";
} else if (elo > 4200 && elo <= 4350) {
return "15k";
} else if (elo > 4350 && elo <= 4500) {
return "14k";
} else if (elo > 4500 && elo <= 4650) {
return "13k";
} else if (elo > 4650 && elo <= 4800) {
return "12k";
} else if (elo > 4800 && elo <= 4950) {
return "11k";
} else if (elo > 4950 && elo <= 5100) {
return "10k";
} else if (elo > 5100 && elo <= 5250) {
return "9k";
} else if (elo > 5250 && elo <= 5400) {
return "8k";
} else if (elo > 5400 && elo <= 5550) {
return "7k";
} else if (elo > 5550 && elo <= 5700) {
return "6k";
} else if (elo > 5700 && elo <= 5850) {
return "5k";
} else if (elo > 5850 && elo <= 6000) {
return "4k";
} else if (elo > 6000 && elo <= 6150) {
return "3k";
} else if (elo > 6150 && elo <= 6300) {
return "2k";
} else if (elo > 6300 && elo <= 6450) {
return "1k";
} else if (elo > 6450 && elo <= 6600) {
return "1d";
} else if (elo > 6600 && elo <= 6800) {
return "1.5d";
} else if (elo > 6800 && elo <= 7000) {
return "2d";
} else if (elo > 7000 && elo <= 7200) {
return "2.5d";
} else if (elo > 7200 && elo <= 7400) {
return "3d";
} else if (elo > 7400 && elo <= 7600) {
return "3.5d";
} else if (elo > 7600 && elo <= 7800) {
return "4d";
} else if (elo > 7800 && elo <= 8000) {
return "4.5d";
} else if (elo > 8000 && elo <= 8200) {
return "5d";
} else if (elo > 8200 && elo <= 8400) {
return "5.5d";
} else if (elo > 8400 && elo <= 8600) {
return "6d";
} else if (elo > 8600 && elo <= 8800) {
return "6.5d";
} else if (elo > 8800 && elo <= 8950) {
return "7d";
} else if (elo > 8950 && elo <= 9100) {
return "1P";
} else if (elo > 9100 && elo <= 9150) {
return "2P";
} else if (elo > 9150 && elo <= 9200) {
return "3P";
} else if (elo > 9200 && elo <= 9250) {
return "4P";
} else if (elo > 9250 && elo <= 9300) {
return "5P";
} else if (elo > 9300 && elo <= 9350) {
return "6P";
} else if (elo > 9350 && elo <= 9400) {
return "7P";
} else if (elo > 9400 && elo <= 9450) {
return "8P";
} else if (elo > 9450 && elo <= 10000) {
return "9P";
} else if (elo > 10000) {
return "AI";
}
return "Unknown";
}

some2112 · 2023-12-01T12:39:30Z

some2112 · 2023-12-01T12:40:55Z

I think the current statistics are very useful, but it may be helpful to evaluate the level if we can have the values of the three largest score losses of Black and White.

kaorahi · 2023-12-02T11:26:26Z

Are you asking me to import your code into this repository? If that's the case, I hesitate because I'm uncertain about the feasibility of your rank estimator, and I'm generally not planning to add new features aggressively here.

Rank estimation does not seem easy. KaTrain once included a rank estimation feature in v1.3.1, which was removed in v1.5 with the note: "The rank estimator has been put away, as it was causing more confusion than anything ..."

some2112 · 2023-12-03T13:35:35Z

Thank you very much for your previous help.

Maybe someone with better go level can find a way to find a reasonable algorithm?
I'm very curious about the author's current go level.

I am a Hong Kong amateur 5d.

kaorahi added a commit to kaorahi/lizgoban that referenced this issue Nov 18, 2023

warn low policy moves (kaorahi/lizzie#7)

6c62f99

kaorahi added a commit that referenced this issue Nov 25, 2023

Implement #7 (intuition misses)

d8efbfc

kaorahi added a commit that referenced this issue Nov 27, 2023

Add "skip-first-moves" to #7 (intuition misses)

c768458

kaorahi changed the title ~~Add 2 new statistical methods~~ Show counts of intuition misses Nov 27, 2023

kaorahi added a commit to kaorahi/lizgoban that referenced this issue Feb 14, 2024

warn overlooked high-policy best moves (kaorahi/lizzie#7)

70ac6f3

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Show counts of intuition misses #7

Show counts of intuition misses #7

some2112 commented Nov 18, 2023

kaorahi commented Nov 18, 2023

kaorahi commented Nov 18, 2023

some2112 commented Nov 18, 2023

some2112 commented Nov 18, 2023

kaorahi commented Nov 20, 2023

some2112 commented Nov 21, 2023 •

edited

Loading

kaorahi commented Nov 23, 2023

some2112 commented Nov 24, 2023

kaorahi commented Nov 25, 2023

some2112 commented Nov 26, 2023

some2112 commented Nov 27, 2023

kaorahi commented Nov 27, 2023

some2112 commented Dec 1, 2023

some2112 commented Dec 1, 2023

some2112 commented Dec 1, 2023

kaorahi commented Dec 2, 2023

some2112 commented Dec 3, 2023

Show counts of intuition misses #7

Show counts of intuition misses #7

Comments

some2112 commented Nov 18, 2023

kaorahi commented Nov 18, 2023

kaorahi commented Nov 18, 2023

some2112 commented Nov 18, 2023

some2112 commented Nov 18, 2023

kaorahi commented Nov 20, 2023

some2112 commented Nov 21, 2023 • edited Loading

kaorahi commented Nov 23, 2023

some2112 commented Nov 24, 2023

kaorahi commented Nov 25, 2023

some2112 commented Nov 26, 2023

some2112 commented Nov 27, 2023

kaorahi commented Nov 27, 2023

some2112 commented Dec 1, 2023

some2112 commented Dec 1, 2023

some2112 commented Dec 1, 2023

kaorahi commented Dec 2, 2023

some2112 commented Dec 3, 2023

some2112 commented Nov 21, 2023 •

edited

Loading