r - Slow plotting SpatialPolygonDataFrames on some systems

I have noticed that my personal computer is very very slow plotting SpatialPolygonDataFrames. I made a question about this issue on the general Stack Overflow, and with the answer I got I ended up learning that the problem was not extensive to all machines:

https://stackoverflow.com/questions/43500213/speed-up-time-to-plot-spatialpolygonsdataframe-extremely-long-in-r

On that question I included the code I was using and the .shp file I am trying to plot. The plot took up to three hours in my machine (Macos), while tests on other computers (windows-based) had the plot in half a minute. Other operations with spatial data work fine on my computer, the issue is with plotting. It happens both when using regular raster plot and +sp.polygons from the rasterVis library.

However, I have not discovered yet what makes it so slow to plot on my machine, and I am thinking that probably other people are in the same situation as me. For this reason, I was hoping that asking the question here could help ,e solve the issue. I am working on a Macbook Pro (Retina, 13-inch, Late 2013) updated to the last SO, Sierra. This is my sessionInfo():

> sessionInfo()
R version 3.3.2 (2016-10-31)
Platform: x86_64-apple-darwin13.4.0 (64-bit)
Running under: macOS Sierra 10.12.4

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] parallel  splines   stats     graphics  grDevices utils     datasets  methods  

[9] base     

other attached packages:
 [1] rasterVis_0.41      latticeExtra_0.6-28 RColorBrewer_1.1-2  rgdal_1.2-6        
 [5] dismo_1.1-4         ecospat_2.1.1       gbm_2.1.3           lattice_0.20-35    
 [9] survival_2.41-3     ape_4.1             ade4_1.7-6          raster_2.5-8       
[13] sp_1.2-4            dplyr_0.5.0         purrr_0.2.2         readr_1.1.0        
[17] tidyr_0.6.1         tibble_1.3.0        ggplot2_2.2.1       tidyverse_1.1.1    

loaded via a namespace (and not attached):

  [1] TH.data_1.0-8         colorspace_1.3-2      deldir_0.1-12        
  [4] class_7.3-14          htmlTable_1.9         base64enc_0.1-3      
  [7] hexbin_1.27.1         MatrixModels_0.4-1    earth_4.4.9.1        
 [10] mvtnorm_1.0-6         lubridate_1.6.0       xml2_1.1.1           
 [13] codetools_0.2-15      mnormt_1.5-5          doParallel_1.0.10    
 [16] knitr_1.15.1          polyclip_1.6-1        Formula_1.2-1        
 [19] jsonlite_1.4          mda_0.4-9             pROC_1.9.1           
 [22] broom_0.4.2           cluster_2.0.6         httr_1.2.1           
 [25] backports_1.0.5       assertthat_0.2.0      Matrix_1.2-8         
 [28] lazyeval_0.2.0.9000   adehabitatHR_0.4.14   acepack_1.4.1        

 [31] htmltools_0.3.5       quantreg_5.33         tools_3.3.2          
 [34] NLP_0.1-10            ecodist_1.2.9         gtable_0.2.0         
 [37] reshape2_1.4.2        Rcpp_0.12.10          spatstat_1.50-0      
 [40] PresenceAbsence_1.1.9 slam_0.1-40           cellranger_1.1.0     
 [43] nlme_3.1-131          iterators_1.0.8       psych_1.7.3.21       
 [46] stringr_1.2.0         rvest_0.3.2           gtools_3.5.0         
 [49] goftest_1.1-1         polspline_1.1.12      MASS_7.3-45          
 [52] zoo_1.8-0             scales_0.4.1          hms_0.3              
 [55] spatstat.utils_1.4-1  sandwich_2.3-4        SparseM_1.76         
 [58] gridExtra_2.2.1       TeachingDemos_2.10    rms_5.1-0            

 [61] rpart_4.1-10          reshape_0.8.6         stringi_1.1.5        
 [64] maptools_0.9-2        foreach_1.4.3         plotrix_3.6-4        
 [67] randomForest_4.6-12   permute_0.9-4         e1071_1.6-8          
 [70] checkmate_1.8.2       boot_1.3-19           tensor_1.5           
 [73] htmlwidgets_0.8       biomod2_3.3-7         plyr_1.8.4           
 [76] magrittr_1.5          R6_2.2.0              Hmisc_4.0-2          
 [79] multcomp_1.4-6        DBI_0.6-1             haven_1.0.0          
 [82] foreign_0.8-67        mgcv_1.8-17           maxent_1.3.3.1       
 [85] abind_1.4-5           nnet_7.3-12           modelr_0.1.0         
 [88] MigClim_1.6           grid_3.3.2            readxl_1.0.0         

 [91] data.table_1.10.0     vegan_2.4-3           forcats_0.2.0        
 [94] plotmo_3.3.2          CircStats_0.2-4       classInt_0.1-24      
 [97] digest_0.6.12         adehabitatMA_0.3.11   tm_0.7-1             
[100] munsell_0.4.3         adehabitatLT_0.3.21   viridisLite_0.2.0

I know I have XQuartz installed, but haven't really understand completely how do all that part of R works.