How do I loop through items in a list box and then remove those item?

Do you want to remove all items? If so, do the foreach first, then just use Items.Clear() to remove all of them afterwards.

Otherwise, perhaps loop backwards by indexer:

listBox1.BeginUpdate();
try {
  for(int i = listBox1.Items.Count - 1; i >= 0 ; i--) {
    // do with listBox1.Items[i]

    listBox1.Items.RemoveAt(i);
  }
} finally {
  listBox1.EndUpdate();
}