Still, Iceland will probably not withdraw its application for membership even if the fishing talks are not fruitful, Skarphedinsson said.
"At this point in time, the Icelandic people are not ready to commit to the EU because they want to see the outcome of negotiations ... related to fisheries," he said, Reuters reported.
Iceland began its EU talks this week, launching a process that could take years. The country is up against possible hurdles over its debts, fishing quotas and access for foreign investors to its markets.
The country is up against a strong fisheries lobby that is combating the surrender of its exclusive access to fishing grounds to other states, reported BBC.
Talks commenced on four of the more than 30 policy areas explored in the accession process aimed at synchronizing national laws with EU rules.
Unfortunately for the North Atlantic island nation, disputes over fishing issues have intensified this year. Brussels grew irate when Iceland upped its mackerel catch to take advantage of heftier stocks in its waters.
The EU responded by threatening to block Icelandic vessels from its ports.
"It all hinges on fisheries. If they (EU officials) listen to our arguments carefully and accept our arguments ... it will be swift," Skarphedinsson stated.
"My logic tells me we will come to a deal ... But based on my experience, those negotiations will be difficult, drawn out and might postpone the final outcome ... I assume fisheries will be the last chapter to be finished," he commented.
EU fishery "superpowers" such as Spain will ultimately decide on Iceland's membership, he said, EUobserver reported.
If the talks go well, the bid will still face a referendum.
He said Iceland will probably not be able to join the EU for at least two years.
Although Skarphedinsson appeared confident the mackerel issue would be resolved, he said Iceland had made the right choice increasing its catch because the climbing mackerel stocks jeopardise other regional species.