It may be possible, if in the select event you add the newly selected item to the setSelectedIndices array. You need to keep track of the currently selected indices though, and you need to check whether the selected item is actually selected or unselected using the isIndexSelected method.
Of course, a much better approach would be to use the sap.m.List controls which has much better mobile multiselect support 